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METHODS OF INTERACTING WITH AUDIO 
AND AMBIENT MUSIC 

RELATED APPLICATION DATA 

5 

This application claims the benefit of application 60/1 34, 
782, filed May 19, 1999, attached hereto as Appendix A. 

The technology detailed in the present application is also 
related to that detailed in application Ser. No. 09/343,104, 
filed Jun. 29, 1999; Ser. No. 09/292,569, filedApr. 15, 1999 10 
(now abandoned in favor of continuation application Ser. No. 
10/379,393, filed Mar. 3, 2003, now Pat. No. 7,263,203); Ser. 
No. 09/314,648, filedMay 19, 1999 (now U.S. Pat No. 6,681, 
028); 60/141,763, filed Jun. 30, 1999; 60/158,015, filed Oct. 
6, 1999; 60/163,332, filed Nov. 3 t 1999; 60/164,619, filed 
Nov. 10, 1999; Ser. No. 09/452,023, filedNov. 30, 1999(now 
U.S. Pat. No. 6,408,082); 09/452,021, filed Nov. 30, 1999; 
and in U.S. Pat. No. 5,862,260. 

INTRODUCTION 

20 

16 year old Bob struts into the coffee shop down from high 
school with a couple of buddies, a subtle deep pound in the 
ambient sound track lets them know they 're in the right place . 
The three of them instinctually pull out of their pockets their 
audio Birddawgs (a small hand held unit about the size and 25 
style of an auto-door-alann device, or "fob"), and when they 
see the tiny green light, they smile, high five, and push the big 
"GoFetch" button in synchrony. That tune will now be wait- 
ing for them at home, safely part of their preferred collection 
and ever- so -thankfully not lost to their collective bad memory 30 
(if they even knew the name of the artist and tune title in the 
first place!). 

33 year old Mary is at home listening to the latest batch of 
holiday tunes being offered up over her 2-decade-long favor- 
ite radio station. She *s spent many days now half-consciously 
culling the tunes for that perfect arrangement for the new 
year's bash that she regrettably agreed to host. 10:40 AM rolls 
around and some new tune catches her ear, a tune she knows 
can work well following the jingle-cats rendition of Straw- 
berry Fields. She half jogs over to the stereo and hits the 
"GoFetch" button. In a few days, she'll sit down at the com- 40 
puter and put together the final sound track for the gala 
evening ahead, her play fist dutifully waiting for her shuffling 
instructions and desired start time, 

49 year old Jack (the financial analyst) is thoroughly bored 
sitting in the crowded gate D23 at Dulles, Droning 20 feet up 45 
and over his head is the airport network station, currently 
broadcasting the national weather report. As the segue to the 
business segment approaches, the teaser review mentions that 
they'll be having a report on today's rally in the bond market 
and the driving forces behind it. Jack puis out his Birddawg- 50 
enabled Palm Pilot on the off-chance they actually will have 
a little depth in the reporting. Indeed, as the segment plays and 
starts discussing the convoluted effects of Greenspan's 
speech to the Intemet-B-Free society, he taps the "GoFetch" 
button, knowing that once he gets back to his main browsing 
environment he will be able to follow dozens of links that the 55 
airport network has pre-assigned to the segment. 

The foregoing and other features and advantages of the 
present invention will be more readily apparent from the 
following detailed description, which proceeds with refer- 
ence to the accompanying figures. so 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a diagram showing the participants, and channels, 
involved in the distribution of music. 65 

FIG. 2 shows a conceptual model of how music artists, 
record labels, and E-Music distributors can all interact with a 


Media Asset Management System, of which several are 
detailed in the following specification. 

FIG. 3 is a block diagram of a device according to one 
embodhnent of the present invention. 

FIG. 4 is a block diagram of a system in which the device 
of FIG. 3 may be utilized. 

DETAILED DESCRIPTION 

Referring to FIG. 3> a device 10 according to one embodi- 
ment of the present invention includes a microphone 12, an 
A/D converter 13, a processor 14, one or more indicators 16, 
one or more buttons 18, a wireless interface 20, and a power 
source 22. 

The device can be packaged in a small plastic housing, 
preferably as small as is practical (e.g., sized and configured 
to serve as a key chain ornament, perhaps akin to the Toma- 
gatchi toys that were recently popular). The housing has one 
or more small holes to permit audio penetration through the 
housing to the microphone 12. 

The processor 14 can take various forms, including a dedi- 
cated hardware device (e.g., an ASIC), a general purpose 
processor programmed in accordance with instructions stored 
in non- volatile RAM memory, etc. 

The indicators 1 6 can be as simple as a single LED lamp, or 
as complex as an alphanumeric LCD or other multi-element 
display, hi one embodiment, the indicator simply indicates 
when the processor has decoded a watermark in audio sensed 
by the microphone. More elaborate signaling techniques can 
of course be used, including two- or three-color LEDs that 
can be used to signal different states with different colors, 
indicators with flashing patterns or changing displays, etc. 

The buttons 18 are used by the user to indicate an interest in 
the audio just-heard. In one embodiment, there is a single 
button 18, and it is emblazoned with a stylized legend that can 
serve as a trademark or service mark, e.g., GetTt!, GoFetch, 
Birddawg, something Batrnan-esque ("Wham," "Zappp," 
"Pow! !," etc.), or something more mundane (e.g., Capture). 

r Fhe power source 22 can be a battery, solar cell, storage 
capacitor, or other source of energy suitable for powering the 
components of the device 10. 

The wireless interface 20 serves to exchange data with a 
relay station 24 (FIG. 4). In one embodiment, the interface is 
radio-based, and provides a one-way communications chan- 
nel. In other embodiments other wireless technologies can be 
used (e.g., IR), and/or two-way communication can be pro- 
vided. 

The relay station can be a cellular repeater (if the interface 
transmits using cellular frequencies and protocols), or a beat 
receiver, e.g., associated with the user's computer. The relay 
station can also be a paging system relay station (e.g., as are 
used for two-way pagers), or may be a low earth orbit satel- 
lite-based repeater. 

In operation, the processor monitors the ambient audio for 
the presence of encoded data, e.g., a digital watermark, and 
decodes same. If power considerations permit, the device is 
"always-on " In other embodiments, one of the buttons 18 can 
be used to awaken the device. In such other embodiments* 
another button-press can serve to turn-off the device, or the 
device can power-down after a predetermined period, e.g., of 
not sensing any watermarked audio. 

A number of techniques for watermarking audio (and 
decoding same) are known, as illustrated by U.S. Pat. Nos. 
5,862,260, 5,963,909, 5,940,429, 5,940,135, 5,937,000, 
5,889,868, 5,833,432, 5,945,932, W09939344 (correspond- 
ing to U.S. application Ser. No. 09/017,145, now U.S. Pat, 
No. 6,145,081), and W09853565 {corresponding to U.S. 
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application Ser. Nos. 08/858,562 and Ser. No. 08/974,920, 
now U.S. Pat. Nos. 5,940,135 and 6,175,627, respectively). 
Commercially-available audio watermarking software 
includes that available from AudioTrack, Verance (formerly 
Aris/Solana), Cognicity, Liquid Audio, and others. 5 

The data payload encoded by the watermark (the audio-ID) 
may take various forms. One is a Digital Object Identifier — 
an ID corresponding to the standardized digital object num- 
bering system promulgated by the International DO I Foun- 
dation (www.doi.org). Another is to include plural data fields 1 0 
variously representing, e.g., the name of the publisher, the 
name of the artist, the title of the work, the date of publication, 
etc., etc. Another is to encode a unique identifier (UID), e.g., 
o f 1 6-64 bits. The UID serves as an index to a remote database 
where additional information (e,g„ publisher, artist* title, date 1 5 
of publication, etc, > are stored). The data transmitted from the 
device 10 to the relay station 24 typically includes some or all 
of the watermark payload data, and also includes data iden- 
tifying the device 10, or its user (user- ED data). Again, this 
data can include several data fields (e.g. user name, audio 20 
delivery information such as email address or URL, age, 
gender, model of device 10, etc.). Alternatively, a serial num- 
ber or other unique identifier can be used, which serves as an 
index to a database have a corresponding record of informa- 
tion relating to the user and/or device. 25 

The audio- ID and user- ID data are typically formatted and 
encoded by the device 10 according to a protocol that pro- 
vides error correcting, framing, and other data useful in assur- 
ing reliable transmission to the relay station, and/or for fur- 
ther transport. 30 

Some embodiments of device 10 recognize just a single 
form of watermarking, and can understand only payload data 
presented in a single format. In other embodiments , the 
device may be capable of recognizing watermarking accord- 35 
ing to several different techniques, and with several different 
payload formats. This latter functionality can be achieved, 
e.g., by cyclically trying different decoding techniques until 
one that produces valid output data (e.g., by reference to a 
checksum or other indicia) is obtained. That decoding tech- 40 
nique and payload interpretation can thereafter be used until 
valid output data is no longer obtained. 

bi some embodiments, the device 10 transmits data to the 
relay station at the moment the user presses the button 18. In 
other embodiments, a store- and- forward mode is used, That 45 
is, when the user presses the button 18, the decoded water- 
mark data is stored in memory within the device. Thereafter, 
e.g., when the device is coupled with a "nest" or "holster*' at 
the user's computer (or when download capability is other- 
wise activated), the stored data is downloaded— either 30 
through mat device or otherwise. 

The infrastructure between the device 10 and delivery of 
the audio to its ultimate destination can take myriad forms. 
One is shown in FIG . 4. In this arrangement, some or all of the 
data received by the relay station 24 is routed through the 55 
internet 26 to a server 28. (The server 28 can be a "Medi- 
aBridge" server of the type described, e.g., in the assignee's 
application 60/164,619, filed Nov 10, 1999, and Ser. No. 
09/343 , 1 04, filed Jun. 29, 1 9 99 .) Server 28 parses the data and 
routes some or all of it to a data repository 30 at which the eo 
audio requested by the user is stored. This repository, in turn, 
dispatches the audio to the user (e.g., to a computer, media 
player, storage device, etc.), again through the internet. (Ad- 
dress information detailing the destination 32 of the audio 
may be included in the data sent from the device 10, or can be es 
retrieved from a database at the server 28 based on a user- ID 
sent from the device 10.) 


In some embodiments, the repository 30 (which may be 
co- located with server 28, or not) includes various data 
beyond the audio itself. For example, the repository can store 
a collection of metadata (e.g., XML tags) corresponding with 
each stored item of audio. This metadata can be transmitted to 
the user's destination 32, or can be used, e.g., for rights 
management purposes (to limit the user's reproduction or 
re-distribution rights for the audio, etc.), to establish a fee for 
the audio, etc. One suitable metatag standard is that under 
development by <indecs> (Interoperability of Data in 
E-Commerce Systems, www.indecs,org). 

The audio data can be delivered in streaming form, such as 
using technology available from RealNetworks (RealAudio), 
Microsoft (Windows Media Player), MP3, Audiobase, Beat- 
nik, Bluestreak.com, etc. The former three systems require 
large (e,g., megabytes) player software on the receiving (cli- 
ent) computer* the latter do not but instead rely, e.g., on small 
Java applets that can be downloaded with the music. 

Alternatively, the audio can be delivered in a file format. In 
some embodiments the file itself is delivered to the user's 
destination 32 (e_g.> as an email attachment). In others, the 
user is provided a URL to permit access to, or downloading 
of, the audio. (The URL may be a web site that provides an 
interface through which the user can pay for the requested 
music* if pre-payment hasn't been arranged.) 

The user's destination 32 is typically the user *s own com- 
puter. If a "live" IP address is known for that computer (e,g., 
by reference to a user profile database record stored on the 
server 28), the music can be transferred immediately. If the 
user's computer is only occasionally connected to the inter- 
net, the music can be stored at a web site (e.g. protected with 
a user- set password)* and can be downloaded to the user's 
computer whenever it is convenient. 

In other embodiments, the destination 32 is a personal 
music library associated with the user. The library can take the 
form, e.g., of a hard-disk or semiconductor memory array in 
which the user customarily stores music. This storage device 
is adapted to provide music data to one or more playback units 
employed by the user (e.g. a personal MP3 player, a home 
stereo system, a car stereo system, etc.). In most installations, 
the library is physically located at the user's residence, but 
could be remotely sited, e.g. consolidated with the music 
libraries of many other users at a central location. 

The personal music library can have its own internet con- 
nection. Or it can be equipped with wireless capabilities, 
pennitting it to receive digital music from wireless broadcasts 
(e.g. from a transmitter associated with the server 28). In 
either case, the library can provide music to the user's play- 
back devices by short-range wireless broadcast. 

In many embodiments, technology such as that available 
from Sonicbox, permits audio data delivered to the computer 
to be short range FM-broadcast by the user's computer to 
nearby FM radios using otherwise- unused radio spectrum. 

Some implementations of the present invention support 
several different delivery technologies (e.g., streaming, file, 
TJRL), and select among them in accordance with the profiles 
of different users. 

Payment for the audio (if needed) can be accomplished by 
numerous means. One is by charging of a credit card account 
associated with the user (e.g., in a database record corre- 
sponding to the user-ID). 

Some implementations of the invention make use of secure 
delivery mechanisms, such as those provided by InterTrust, 
Preview Systems, etc. In addition to providing secure con- 
tainers by which the audio is distributed, such systems also 
include their own secure payment facilities. 
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By such arrangements, a user can conveniently compile an 
archive of favorite music — even while away from home. 

To provide a comprehensive disclosure without unduly 
lengthening this specification, the disclosures of the applica- 
tions and patents cited above are incorporated herein by ref- s 
ereuce. 

Having described and illustrated the principles of my 
invention with reference to a preferred embodiment and sev- 
eral variations thereof, it should be apparent that the detailed 
embodiment is illustrative only and should not be taken as 1 0 
limiting the scope of my invention. 

For example, while the invention is illustrated with refer- 
ence to a button that is activated by the user to initiate capture 
of an audio selection, other interfaces can be used. For 
example, in some embodiments it can be a voice-recognition 1 5 
system that responds to spoken commands, such as "capture" 
or Record." Or it can be a form of gesture interface. 

Likewise, while the invention is illustrated with reference 
to a stand-alone device, the same functionality can be built- 
into radios (including internet- ba sed radio s that receive w ire- 20 
less IP broadcasts) , computer audio systems, and other appli- 
ances. In such case the microphone can be omitted and, in 
some cases, the wireless interface as well. (The data output 
from the device can be conveyed, e.g., through the network 
connection of an associated computer, etc) 25 

Moreover, while the invention is illustrated with reference 
to an embodiment in which audio, alone, is provided to the 
user, this need not be the case. As in the Dulles airport sce- 
nario in the introduction, the server 28 can provide to the user 3Q 
several internet links associated with the sensed audio. Some 
of these links can provide commerce oprxirtunities (e.g., to 
purchase a CD on which the sensed audio is recorded). Others 
can direct the user to news sites, concert schedules, fen-club 
info, etc. In some such embodiments, the ancillary in forma- 35 
tion is provided to the user without the audio itself 

Although not particularly detailed, the data provided to the 
user^s destination typically includes information about the 
context in which the data was requested. In a simple case this 
can be the time and date on which the user pressed the Capture 40 
button. Other context information can be the identification of 
other Birddawg devices 10 that were nearby when the Cap- 
ture button was pressed. (Such information can be gleaned, 
e.g., by each device transmitting a brief Who Ami message 
periodically, receiving such messages from other nearby 45 
devices, and logging the data thus received.) 

Still other context information might be the location from 
which the Capture operation was initiated. This can be 
achieved by decoding of a second watermark signal, e.g., on 
a low level white-noise broadcast. The public address system 50 
in public pkces, for example, can broadcast a generally- 
indiscernable noise signal mat encodes a watermark signal- 
Devices 10 can be arranged to detect two (or more) water- 
marks from the same audio stream, e.g., by reference to two 
pseudo-random sequences with which the different water- 55 
marks are encoded. One identifies the audible audio, the other 
identifies the location. By such an arrangement, for example, 
the device 10 can indicate to the server 28 (and thence to the 
user destination 32) the location at which the user encoun- 
tered the audio. (This notion of providing location context 60 
information by subliminal audio that identifies the location 
has powerful applications beyond the particular scenario con- 
templated herein.) 

hi some embodiments, the device 10 can buffer watermark 
information from several previous audio events* permitting 65 
the user to scroll back and select (e.g., in conjunction with a 
screen display 16) the ID of the desired audio. 


An arrangement like the foregoing may require that the 
decoded watermark information be interpreted for the user, so 
that the user is not presented simply a raw binary watermark 
payload. The interpreted information presented to the user 
can comprise, e.g., the source (CNN Airport News, WABC 
Radio, CD-ROM, MTV), the artist (Celine Dion), the title 
(That's the Way It Is), anc7ormetiinedecoded(3:38:02p,m,), 
etc. 

One way to achieve the foregoing functionality is to convey 
both the binary UID payload and abbreviated text (e.g., 5- or 
6-bit encoded) through the watermark "channel" on the 
audio, hi one such arrangement, the watermark channel con- 
veys data a UID, four characters of text, and associated error- 
correcting bits* every ten seconds, In the following ten sec- 
onds the same UID is conveyed, together with the next four 
characters of text. 

Another way to achieve such functionality is to provide a 
memory in the device 10 that associates the watermark pay- 
load (whether UID or field- based) with corresponding textual 
data (e.g the source/artist/title referenced above). A 1 mega- 
byte semiconductor non- volatile RAM memory, for example, 
can serve as a look-up table, matching code numbers to artist 
names and song titles. When the user queries the device to 
learn the identify of a song (e.g., by operating a button IS), the 
memory is indexed in accordance with one or more fields 
from the decoded watermark, and the resulting textual data 
from the memory (e.g. source/artist/title) is presented to the 
user. 

Such a memory will commonly require periodic updating. 
The wireless interface 20 in device 10 can include reception 
capabilities* providing a ready mechanism for providing such 
updated data. In one embodiment, the device "awakens" 
briefly at otherwise idle moments and tunes to a predeter- 
mined frequency at which updated data for the memory is 
broadcast, either in a baseband broadcast channel, or in an 
aQcillary (e.g. SCA) channel. 

Invariants of the foregoing, internet delivery of update data 
for the memory can be substituted for wireless delivery. For 
example, a source/artist/title memory in the device 10 can be 
updated by placing the device in a '"nest" every evening. The 
nest (which may be integrated with a battery charger for the 
appliance) can have an internet connection, and can exchange 
data with the device by infrared, inductive, or other proxim- 
ity-coupling technologies, or through metal contacts. Each 
evening, the nest can receive an updated collection of source/ 
artist/title data, and can re-write the memory in the device 
accordingly. By such arrangement, the watermark data can 
always be properly interpreted for presentation to the user. 

The "Capture" concepts noted above can be extended to 
other functions as well. One is akin to forwarding of email, If 
a consumer hears a song that another friend would enjoy, the 
listener may send a copy of me song to the friend. This 
instruction can be issued by pressing a "Send" button, or by 
invoking a similar function on a graphical (or voice- or ges- 
ture-responsive) user interface. In response, the device so- 
instructed can query the person as to the recipient. The person 
can designate the desired recipient(s) by scrolling through a 
pre-stored list of recipients to select the desired one. (The list 
can be entered through a computer to which the device is 
coupled.) Alternatively, the user can type-in a name (if the 
device provides a keypad), or a portion thereof sufficient to 
uniquely identify the recipient. Or the person may speak the 
recipient's name. As is conventional with hands- free vehicle 
cell phones, a voice recognition unit can listen to the spoken 
instructions and identify the desired recipient. An "address 
book"-like feature has the requisite information forthe recipi- 
ent (e. g., the web site, IP address, or other data identifying the 
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location to which music for that recipient should stored or 
queued, the format in which the music should be delivered, 
etc) stored therein. In response to such command, the appli- 
ance dispatches instructions to the server 28, including an 
authorization to incur any necessary charges (e.g., by debiting 
the sender's credit card). Again, the server 28 attends to 
delivery of the music in a desired manner to the specified 
recipient. 

Still further, a listener may query the device (by voice, GUI 
or physical buttons textual, gesture, or other input) to identify 
CDs on which the ambient audio is recorded. Or the listener 
may query the device for the then- playing artist's concert 
schedule. Again, the appliance can contact a remote database 
and relay the query, together with the user ID and audio ID 
data. The database locates the requested data, and presents 
same to the user — either through a UI on device 10, or to the 
destination 32. If desired, the user can continue the dialog 
with a further instruction, e.g., to buy one of the CDs on which 
the then-playing song is included. Again, this instruction may 
be entered by voice* GUI, etc, and dispatched from the device 
to the server, which can then complete the transaction in 
accordance with pre- stored information (e.g. credit card 
account number, mailing address, etc.). A confirming mes- 
sage can be relayed to the device 10 or destination 32 for 
presentation to the user. 

While the invention particularly contemplates audio, the 
principles detailed above find applications in many other 
media, and in many other applications of the MediaBridge 
server 28. 

Moreover, while the invention particularly contemplates 
watermarks as the channel by which audio is identified, in 
other embodiments different tectmiques can be used. For 
example, digital radio protocols provide ID fields by which 
audio can be identified. Similarly, IP protocols for internet 
delivery of radio include identification fields within their 
packet formats. Accordingly, audio distributed according to 
formats that include audio IDs therein can likewise be 
employed according to the present invention. 

In view of the many embodiments to which the principles 
o four invention may be applied, it should be apparent that the 
detailed embodiment is illustrative only and should not be 
taken as limiting the scope of the invention. Rather, we claim 
as our invention all such modifications as may fall within the 
scope and spirit of the following claims, and equivalents 
thereto. 

APPENDIX A 

Application 60/134,782, filed May 19, 1999 

METHODS AND SYSTEMS EMPLOYING 
DIGITAL WATERMARKING 

FIELD OF THE INVENTION 

The present invention relates to applications of digital 
watermarking in conjunction with audio, video, imagery, and 
other media content. 

BACKGROUND 

Watermarking (or "digital watermarking") is a quickly 
growing field of endeavor, with several different approaches. 
The present assignee's work is reflected in U.S. Pat. Nos. 
5,841,978, 5,768,426, 5,748,783, 5,748,763, 5,745,604, 
5,710,834, 5,636,292, 5,721,788, and laid-open PCT appli- 
cations WO 97/43736 and WO 99/10837. Other work is illus- 
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txated by US. Pat. Nos. 5,734.752, 5,646,997, 5,659,726, 
5,664,018, 5,671,277, 5,687,191, 5,687,236, 15 5,689,587, 
5,568,570, 5,572,247, 5,574,962, 5,579,124, 5,581,500, 
5,613,004, 5,629,770, 5,461,426, 5,743,631, 5,488,664, 

5 5,530,759,5,539,735, 4,943,973, 5,337,361, 5,404,160, 
5,404,377, 5,315,098, 5,319,735, 5,337,362, 4,972,471, 
5,161,210, 5,243,423, 5,091,966, 5,113,437,4,939,515, 
5,374,976, 4,855,827, 4,876,617, 4,939,515, 4,963,998, 
4,969,041 , and published foreign applications WO 98/02864, 

10 EP 822,550, WO 97/39410, WO 96/36163, GB 202,196,167, 
EP 777,197, EP 736,860, EP 705,025, EP 766,468, EP 782, 
322, WO 95/20291, WO 96/26494, WO 96/36935, WO 
96/42151, WO 97/22206, WO 97/26733. 
Most of the work in watermarking, however, is not in the 

i s patent literature but rather in published research. In addition 
to the patentees of the foregoing patents, some of the other 
workers in this field (whose watermark-related writings can 
by found by an author search in the INSPEC database) 
include I. Pitas, Eckhard Koch, Jian Zhao, Norishige 

20 Morimoto, Laurence Boney, Kineo Matsui, A Z. Tirkel, Fred 
Mintzer, B. Macq, Ahmed H. Tewfik, Frederic Jordan, Nao- 
hisa Komatsu, and Lawrence O 'Gorman. 

The artisan is assumed to be familiar with the foregoing 
prior art. 

25 In the present disclosure it should be understood that ref- 
erences to watermarking encompass not only the assignee's 
watermarking techno logyj but can likewise be practiced with 
any other watermarking technology, such as those indicated 
above. 

30 Watennarking has various uses, but the present specifica- 
tion details several new uses that provide Junctional ity and 
features not previously available. 

35 BRIEF DESCRIPTION OF THE DRAWINGS 

FIG, 1 is a diagram showing the participants, and channels, 
involved in the distribution of music. 

FIG. 2 shows a conceptual model of how music artists, 
40 record labels, and E-Music distributors can all interact with a 
Media Asset Management System, of which several are 
detailed in the following specification. 

DETAILED DESCRIPTION 

For expository convenience, much of the following discus- 
sion focuses on music, but the same principles and techniques 
are largely or wholly applicable to other source data, whether 
non-music audio, video, still imagery, printed materials, etc. 

50 

Music Asset Management 

Referring to the figures, the music distribution process 
begins with a creative artist 10, The artist's music has tradi- 
tionally been distributed by a record label 12. (While the 

55 following discussion refers to distribution through such a 
label, it should be understood that such distribution can just as 
well be effected directed under the artist's control, without a 
record label intermediary.) 
In traditional distribution 14, the record label produces 

so tangible media, such as records, tapes, videos (e.g, music 
videos), and CDs 16. These media are physically distributed 
to end-consumers 18. Additionally, the label 12 distributes the 
music media to outlets 20, such as radio and TV stations* 
cable and satellite systems, etc., which broadcast (or narrow- 

65 cast) the artist's work to an audience. Distribution through 
such media outlets may be monitored by playout tracking 
services. Playout traclcing data, collected by firms including 
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Arbitron, Nielsen, ASCAP, BMT, etc., can be used to com- a voluntary specification that will assure proper compensa- 

pute royalty payments > to verify broadcast (e.g. for advertis- tion to those who produce and distribute music. One element 

ing), etc, o f the system will likely be a watermarking component. (Dow 

Increasingly, the distribution of the music to the media Jones Newswire, "Spurred By Maverick Technology, Music 

outlets is performed electronically, Such distribution first $ Industry Eyes Web, 1 ' Dec. 31, 1998.) 

took the form of analog audio over high quality landlines or Yet another initiative has been announced by Solana and 

satellite channels. Digital audio quickly supplanted analog ASCAP. Other companies promoting watermarking for 

audio in such distribution channels due to higher fidelity- music include Aris Technology, MCY.com, and AudioSoft 

More recently, distribution of the music from the record The watermark payload can represent various types of 

labels to the media outlets has occurred over secure links, now 10 d ata . An exemplary payload includes data relating to the 

including the internet. Such security was first provided aim- artist, distribution entity, title, and copyright date/proprietor, 

ply by scrambling the audio signal or data. More sophisti- Additionally, the payload can include a digital Object identi- 

cated "containef-based systems are now coming into vogue, fier- an ISBN- like number issued by a central organization 

in which the audio is "packaged" (often in encrypted form) ( ag , a rights management organization) to uniquely identify 

with ancillary data. 15 the work. 

Electronic distribution of music to the consumer is also SuCD pay j oat i data can be encoded literally (e.g. the title by 

gaining popularity, presently in the MP3 format pnmarily. a series 0 f ASCH characters, etc.). In other embodiments, 

The music providers may deal directly with the public, but codes or abbreviations can be employed with each code hav- 

more commonly effect such consumer distribution through a ing aknown meaning. In still other embodiments, the data can 

newly emerging tier of digital media outlets, such as internet 20 be meaningless by itself, but may serve as a key (e.g., a 

sites that specialize in music. From such sites, consumers can Unique Identifier, or UID) into a remote data database or 

download digital audio fdes into personal digital audio play- repository. An example of such a remote data repository is a 

ers, (The Diamond Rio, and the Audible MoviePlayer web site at a Master Global Address (MGA) associated with 

devices are some of the first of what will doubtless be a large content as detailed below 

number of entrants into mis personal internet audio appliance is M exeffip]arv data payJoad maVj for example, have the 

market.) Or the downloaded data can be stored by the con- foUowin a format - 

sumer-recipient onto any other writeable media (e.g. hard to 

disk, CD, DVD, tape, videotape, etc.). Typically a personal ABCDEFGHI 
computer is used for such downloading, but this intermediary 

may be dispensed with by coupling next generation of per- 30 ™„ Aisask _ lwte / 8 ^ 

sonal audio appliances to an internet-like link. 7- t 1 u * -V ! Z u 1 T^ g 

Thedatod^idoadedhy^ £ a + dlgl ^ L W^ 1 ^ * a 

in me native digital format, translated into another digital Global AMress through a dta&ul name server as 

formatCwhichtranslationmayincludedecrvptionJ^onvetted dlsCuSs f t d ^ '* 1S f a ^-byteASCn field serving as a key 

into analog and recorded in analog form, etc, 35 f?™^* X fieIdofU, f r ™ ^f^^^ 

Unauthorised copying or use of the music can occur any- ASCH field ^rvmg as a key into a title fie d of the remo te 

wheremtheforegomgchannek.However.oneofthegreatest database; D is a 4-bit field servmg as a key mto a label field 

risks occurs once the music has been delivered to the con- of ^ remote database E is an 8-bit integer represen mg the 

sumer (whether by tangible media, by traditional broadcast ^' year fJ^^^ 1 0 T ^ TGS ^ }f 

mediaoutlets^yemergmgdigitaldistributio^orom^ ,0 2000); F is a Infield servmg as akey into a pnee field of 

The general idea of embedding auxiliary data into music * e re ™* f*^: 13 a ™o-byte usage control string 

(i.e. watermarking) has been widely proposed, but so far has (detailed below), H is a streaming data channel and I is a 

been of limited applicability. strm S of bits servmg as a cychc redundancy checksum for the 

For example, GoodNoise is planning to embed a digital Agoing. (More sophisticate^ error correcting checksums 

signature - termed a multimedia identifier, or MMI in its as <»' of be /? pl ^> ^ ^Ti ^ J 

MP3 music, MMI will register the song and its author with a ** ls > exclusive of me CRC coduig and the streaming data 

licensing number, hi addition to providing information about channel. 

me songwriter and distributor, this digital encoding may also ^ Payload is encoded repeatedly, or redundantly 

include lyrics, liner notes, and other information. But all of tough the music, so that the full payload can be decoded 

the proposed uses serve only to convey information from the 50 from P mi ^ excerpts of the music, 

distributor to the consumer, use for "tracking" is actively T° e encoding is also desirably perceptually adaptive, so 

disclaimed. (Wired News, "GoodNoise Tags MP3 Files/T-eh that higher energy encoding is employed where the listener is 

3, 1999 .) less likely to perceive the additional "noise" introduced by the 

The Genuine Music Coalition— a partnership of various encoding, and vice versa. Various techniques for perceptually 

companies in me music distribution business— likewise has 55 adaptive encoding are known. For example, some tie the 

announced plans to employ watermarking 0 f MP 3 music. The amplitude of the encoded signal to the instantaneous ampli- 

watermarking technology, to be provided by Liquid Audio, ^de of the music. Others exploit psychoacoustic "masking" 

will convey data specifying the artist or producer contact of 0Jle ^g^ 1 by a spectrally-or temporally-adjoining signal 

copyright data, and a number to track ownership. The Coali- of higher energy. Still other approaches fill gaps in the 

tion hopes that the provision of mis embedded information eo music's spectrum with watermark energy. These and other 

will help thwart piracy. Industry observers believe Liquid techniques are detailed in the patents incorporated by refer- 

Audio will next introduce playback technology only plays ence, 

audio in which its watermark is detected. (Wired News, "Liq- In other embodiments, perceptually adaptive encoding is 

uefyingMPS, 11 Jan. 23, 1999.) not used. In some such embodiments, no tailoring of the 

A similar initiative has been announced by the Recording 65 temporal or spectral characteristics 0 f the watermark signal is 

Industry Association of America (RIAA). Termed the Secure employed. In others, the watermark signal is spectrally fil- 

Digital Music Initiative (S DMI), the program seeks to define tered to emphasize low frequency audio components (e.g. less 


Page 7 of 20 


US 7,562, 

11 

than 500 hz), high frequency audio components (e.g. higher 
than 2500 hz), or mid- frequency audio components (500- 
2500 hz). 

The streaming data field channel (H) is a medium by which 
data can be conveyed from a distribution site (or other site) to 5 
the end user. Such data may be entirely unrelated to the 
underlying work. For example, it may serve a utilitarian pur- 
pose, such as conveying data to a memory in the consumer 
device to replace previously- stored data that is out-of-date. It 
may be a commercial channel on which bandwidth is sold for to 
access to the consumer or the consumer's device, Essentially 
any purpose can be served by this streaming data field. Unlike 
most of the other fields, the streaming data field may not 
endlessly repeat the same data, but can convey data that 
changes w ith time. i s 

Desirably, the encoding is performed in a manner permit- 
ting recovery of the watermark data even if the audio is 
corrupted, e.g. by format conversion, re-sampling, tape wow 
and flutter, compression, coding, or various forms of audio 
processing (e.g. filtering, pre-emphasis, re-scaling, etc.). One 20 
way to provide for such robustness is to encode a signal of 
known character that can be recognized through all such 
corruption. By identifying such known signal, the watermark 
signal can then be decoded, (The known signal can take 
various forms, e.g. a synchronization signal, a marker signal, 25 
calibration signal, a universal code signal as described in 
applicant's patents, etc.) 

In some embodiments, a watermark "dial-tone" signal is 
provided, This dial-lone signal is a low amplitude, relatively 
wideband, repetitive signal that commonly conveys only lim- 30 
ited information (e.g. a single bit of information). Its presence 
in an audio signal can serve as a "do not record " or similar 
instruction signal. Alternatively, or in addition, the dial- tone 
signal can serve as an aid in "locking" to a plural-bit digital 
watermark signal that is also encoded in the audio. For 35 
example, the cyclical repetition of the signal can serve to 
identify the start of the plural-bit digital watermark signal. Or 
the spectrum or repetition rate of the signal can identify any 
temporal corruption of the audio. An exemplary such signal is 
detailed as a "simple universal code" in Pat. No. 5,636 T 292. 40 

A track of music can be pre-authorized for specified types 
of use. For example, the usage control string of the watermark 
payload may include a six-bit field detailing the classes of 
devices for which the audio is authorised. Each bit would 
correspond to a different class of device. Class 1 devices may 45 
be personal playback devices with only analog-audio output. 
Class 2 devices may be personal entertainment devices 
capable of outputtiug music in digital (e.g. MP3, 
redbook, *.WAV) format, as well as analog audio. Class 3 
devices may be personal computer systems (i.e. with essen- 50 
tially urdimited ability for processing and outputting digital 
audio). Etc., etc. A device to which such MP 3 audio is pro- 
vided would check the usage control string data to determine 
whether it is authorized to utilize the audio. A personal play- 
back device with analog- only output, for example, would 55 
examine the first bit of the usage control string. If it was "1, >? 
the device would be authorized to use (i.e, playback) the MP3 
data; if it was a "0 " the device would refuse to play the music. 

In addition to pre- authorization for certain classes of 
devices, the usage control string can also include bits indicat- 60 
ing the number of permitted playbacks. This data can be 
encoded in bits seven through nine, representing eight possi- 
bilities: 

0 — no playback permitted 

1 — single playback permitted 65 

2— two playbacks permitted 
3 -three playbacks permitted 
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4 — four playbacks permitted 

5 — five playbacks permitted 

6 — 10 playbacks permitted 

7 — unlimited playbacks permitted 

8 — refer to associated data (within the watermark* or 
stored at a remote site) which specifies number of per- 
mitted playbacks. 

The playback device may include a non-volatile store in 
which the number of permitted playba cks is stored for each 
track of music. The device would decrement this number at 
the begmning of each playback. 

The usage control string can also include a two-bit field 
(bits ten and eleven) indicating recording permissions. A 
value of 0 means that data corresponding to the MP3 audio 
(regardless of digital format) should never be made available 
to another digital device. A value of 1 means that the data 
corresponding to the MP3 data may be made available once to 
another digital device. A value of 2 means that the data may be 
made available an unlimited number of times to other digital 
devices. (Value 3 is reserved.) 

Another data field that can be included in an audio water- 
mark is a rating that indicates age-appropriateness. Music 
with violence or sexual themes might be given a rating akin to 
the MPAA "PG-l 3" or "R" rating. Audio appliances may be 
programmed to recogni2e the rating of incoming music v and 
to interrupt playback if the rating exceeds a certain threshold 
setting. Various known techniques can be employed to assure 
that such settings cannot readily be changed, e.g., by juvenile 
listeners. 

Another data field that can be included in an audio water- 
mark is a date field. This field can indicate either the date the 
music was watermarked, or a date in the future on which 
certain rights associated with the music should change. Some 
consumers, for example may not wish to purchase perpetual 
playback rights to certainmusical selections. The right to play 
a selection for 6 months may suffice for many consumers, 
especially ifthe price is discounted in viewof the limited term. 
Such an arrangement would not be wholly disadvantageous 10 
music distributors, since some consumers may end up pur- 
chasing music twice if their initial assessment of a musical 
selection's appeal was too short-sighted. (Naturally, the play- 
back equipment would require a source of real-time clock 
data against which the date field in the watermark can be 
checked to ensure that the playback rights have not yet 
expired,) 

Another of the data fields that can be included in an audio 
watermark specifies technical playback parameters. For 
example, the parameter can cause the playback appliance to 
Qpply a spectral equalization that favors bass frequencies, or 
treble frequencies, or mid-range frequencies; etc. Other pre- 
configured equalization arrangements can similarly be 
invoked responsive to watermark data. Likewise, the param- 
eter can invoke special-effects provided by the playback 
appliance, e.g., echo effects, reverb, etc. (Again, such param- 
eters are usually represented in an abbreviated, coded form, 
and are interpreted in accordance with instructions stored in a 
memory (either in the playback appliance, or linked thereto). 

The same data fields and principles can be applied to non- 
audio content. In video, for example, watermarked data can 
adaptively control the display monitor or playback param- 
eters (e.g., color space) to enhance the viewing experience. 

Music Asset Management/Commerce 

The majority of domestic music piracy is not organized. 
Rather, it is a crime of opportunity and convenience. If the 
crime were made more difficult, the alternative of obtaining a 
copy through legitimate channels would be less onerous. 
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Similarly, if the procedure for obtaining a copy through legiti- (e.g. from the clearinghouse). In either case, the library can 

mate channels were simplified, the incentive for piracy would provide music to the user's playback devices by short-range 

be reduced. Watermarking facilitates both — making the wireless broadcast. 

crime more difficult, and making legitimate music acquisition By such arrangement, a user can conveniently compile an 

easier. 5 archive of favorite music — even while away from home. 

Consider, for example, the pricing o f music in conventional Many variants of the foregoing are of course possible. The 

record stores. A CD (compact disk) may cost S 15, but its sale radio can be a portable unit (e.g. a boombox, a Walkman 

may be driven by just one or two popular songs on the disk. To radio, etc.), rather than an automotive unit. The Ul feature 

obtain these songs, the consumers must purchase the entire employed by the user to initiate capture a musical selection 

disk, with perhaps a dozen songs of no particular interest, io need not be a button (physical or on-screen). For example, in 

This, in essence, is a tying arrangement that benefits the some embodiments it can be a voice-recognition system that 

record labels while prejudicing the consumers. Given these responds to spoken commands, such as "capture" or ^record" 

circumstances, and a ready opportunity to make copies, it is Or it can be a form of gesture interface, 

not surprising that customers sometimes make illicit copies. Instead of decoding the watermark only in response to the 

One classic technique of avoiding purchase of a complete 1 5 user's "capture' 1 command, the radio can decode watermarks 
collection of music, when only one or two songs is desired, is from all received programs, and keep the most recent in a 
to record the music off the radio. While of dubious legality, small FIFO memory. By such arrangement, the user need not 
this technique was popular in the era of combined cassette/ issue the capture instruction wliile the song is playing, but can 
radio players. However, the desired music was sometimes do so even after the song is Finished, 
difficult to encounter in a radio broadcast, and the quality was 20 In some embodiments, data corresponding to the water- 
less than superb. mark can be made available to the user in various forms. For 

The combined cassette/radio player has now evolved into a example, it can be presented to the user on an LCD screen, 

general purpose computer with wide-ranging functionality, identifying the artist and song currently playing. If a corre- 

and other sophisticated devices . Music can be acquired off the spending UI button is activated, the device can so-identify the 

web, and can be recorded in various forms (e.g. in a personal 25 last several selections. Moreover, the data need not be pre- 

MP3 player, stored on a hard disk, stored on a wnteable sented to the user in displayed form; it can be annunciated by 

CD-ROM, played back and recorded on analog cassette, etc., known computer-speech technologies instead, 

etc,). The quality can be quite high, and the erratic broadcast In embodiments in which the watermark does not convey 

time problems of radio broadcasts have been overcome by the ASCII text data, but instead conveys UIDs, or coded abbre- 

web's on-demand delivery mechanisms. (Moreover, the 30 viations, the device must generally interpret this data before 

music can be downloaded in faster- man- realtime, a further presenting it to the user, In an illustrative embodiment, the 

benefit over recording- off- the-air techniques.) device is a pocket-sized FM radio and is equipped with a I 

One hybrid between the new and old is a novel radio (e.g., megabyte semiconductor non-volative RAM memory. The 

for use in a car) that has a "capture" button on the front panel memory includes a data structure that serves as a look-up 

(or other form of user interface, e.g.. a Capture icon on a 33 table, matching code numbers to artist names and song titles. 

GUI). If a user hears a song tiiey want to record and keep, they When the user queries the device to learn the identify of a 

press the Capture button while the song is playing. In song, the memory is indexed in accordance with one or more 

response, the radio device decodes a watermark embedded in fields from the decoded watermark, and the resulting textual 

the music, and thereby knows the identity of the music. The data from the memory (e.g. song title and artist) is annunci- 

radio then makes a wireless transmission identifying the user 40 ated or displayed to the user. 

and the desired song. A local repeater network picks up the In most applications, such memory will require frequent 

wireless signal and relays it (e,g, by wireless rebroadcast, by updating. The RF receiver provides a ready mechanism for 

modem* or other communication medium) to a music clear- providing such updated data. In one embodiment, the radio 

inghouse. The clearinghouse charges the user a nominal fee "awakens" briefly at otherwise idle moments and tunes to a 

(e.g. via a pre-arranged credit card), and queues the music for 45 predetermined frequency at which updated data for the 

download to a predetermined location associated with the memory is broadcast, either in a baseband broadcast channel, 

user. or in an ancillary (e.g. SCA) channel. 

hi one embodiment, the predetermined location is the In variants of the foregoing, internet delivery of updated 

user's own computer. If a "live 11 IP address is known for the memory data can be substituted for wireless delivery. For 

user's computer, the music can be transferred immediately. If 50 example, the artist/song title memory in the personal player 

the user's computer is only occasionally connected to the can be updated by placing the player in a "nest" every 

internet, the music can be stored at a web site (e.g. protected evening. The nest (which may be integrated with a battery 

with a user-set password), and can be downloaded to the charger for the appliance) can have an internet connection, 

user's computer whenever it is convenient. and can exchange data with the personal device by infrared, 

In other embodiments, the predetermined location is a 55 inductive, or other pro?rimiry-coupling technologies, or 

personal music I ibrary maintained by the user. The library can through metal contacts , Each evening, the nest can receive an 

take the form, e.g., of a hard-disk or semiconductor memory updated collection of artists/song titles > and can re- write the 

array in which the user customarily stores music . This storage memory in the personal device accordingly. By such arrange- 

device is adapted to provide music data to one or more play- ment, the watermark data can always be properly intepreted 

back units employed by the user (e.g. a personal MP3 player, 60 for presentation to the user. 

a home stereo system, a car stereo system, etc,)* In most The "Capture" concepts noted above can be extended to 

installations, the library is physically located at the user's other functions as well. One is akin to forwarding of email. If 

residence, but could be remotely sited, e.g. consolidated with a consumer hears a song that another friend would enjoy, the 

the music libraries 0 f many other users at a central location. listener can send a copy o f the song to the fri end. This instruc- 

The personal music library can have its own internet con- tion can be issued by pressing a "Send" button, or by invoking 

nection. Or it can be equipped with wireless capabilities, a similar function on a graphical (or voice- or gesture-respon- 

pennitting it to receive digital music from wireles s broadcasts sive) user interface, hi response, the appliance so- instructed 
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can query the person as to the recipient. The person, can 
designate the desired recipient(s) by typing in a name, or a 
portion thereof sufficient to uniquely identify the recipient. 
Or more typically, the person can speak the recipient's name. 
As is conventional with hands- free vehicle cell phones, a 
voice recognition unit can listen to the spoken instructions 
and identify the desired recipient. An "address book"- like 
feature has the requisite information for the recipient (e.g. , the 
web site, IP address, or other data identifying ihe location to 
which music for that recipient should stored or queued, the 
format in which the music should be delivered, etc.) stored 
therein. In response to such command, the appliance dis- 
patches instructions to the clearinghouse, including an autho- 
rization to debit the sender *s credit card for the music charge. 
Again, the clearinghouse attends to delivery of the music in a 
desired manner to the specified recipient. 

Still further, a listener may query the appliance (by voice, 
GUI or physical button, textual, gesture, or other input) to 
identify CDs on which the then- playing selection is recorded. 
Or the listener may query the appliance for the then-playing 
artist's concert schedule. Again, the appliance can contact a 
remote database, relay the query, and forward data from the 
watermark pay load identifying the artist and/or song title to 
which the query relates. The database locates the requested 
data, and relays same back to the appliance for presentation 
(via a display, by machine speech, or other output) to the user. 
If desired, the user can continue the dialog with a further 
instruction, e.g., to buy one of the CDs on which the then- 
playing song is included. Again, this instruction may be 
entered by voice, GUI, etc., and dispatched from the appli- 
cance to the clearinghouse, which can then complete the 
transaction in accordance with pre- stored information (e.g. 
credit card account number, mailing address, etc.). A con- 
firming messa ge is relayed to the appliance for presentation to 
the user 

Wliile the foregoing transactions require a link to a remote 
site or database, other watermark-based consumer services 
can be provided without such a link. For example, a user can 
query the appliance as to the artist or song- title of the selec- 
tion currently playing. The appliance can consult the embed- 
ded watermark data (and optionally consult a memory to 
determine the textual names associated with coded water- 
mark data), and provide the requested information to the user 
(e.g., by a display, annunciation, or other output). 

The foregoing concepts (e.g. Capture, Send, etc.) can also 
be employed in connection with internet- rather than radio- 
delivery of music, (The following discussion is illustrated 
with reference to the "Capture" function, but it will be rec- 
ognized that the other earlier-discussed features can be simi- 
larly implemented.) 

There are many commercial web sites that sell audio (in 
CD form or otherwise)* and offer limited free music down- 
loads, (or music clips) as an enticement to lure consumers. 
But there are also a great number o f music web sites that have 
no commercial pretense. They are hosted by music lovers 
strictly for the enjoyment of other music lovers. When music 
is downloaded from such a web site, the end-user *s computer 
can analyze the digital data to decode watermark data there- 
from. Again, the user can be presented with a "Capture" 
button that initiates a commercial transaction, by which a 
complete copy of the then-downloaded audio is sent to a 
prearranged storage location, and the user's credit card is 
debited accordingly. This transaction can occur indepen- 
dently of the site from which the music is downloaded (e.g. 
through the clearinghouse referenced above). 

While the "Capture" button can be presented on the web- 
site, this would generally not be in keeping with the non- 
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commercial nature of such web sites. Instead, in an exemplary 
embodiment, the Capture feature is a software program resi- 
dent at the user's computer. When this software program is 
invoked by the user, a socket channel is instantiated between 

5 the user's computer and the clearinghouse over the then- 
existing internet connection. The decoded watermark data 
and user ID is transmitted to the clearinghouse over this 
channel, without hiterrupting the user's other activity (e.g. 
downloading music from the non- commercial web site). In 

id response, the clearinghouse transmits the music to the prear- 
ranged location and attends to billing. 

In some embodiments, a watermark detector is included as 
part of the operating system, and constantly monitors all 
TCP/IP, or other internet, data received by the user's com- 

1 5 puter, for the presence of watermarks. In such case, when the 
Capture feature is invoked, the program examines a memory 
location in which the operating system stores the most-re- 
cently received watermark data. In another embodiment, the 
computer does not monitor all internet traffic for embedded 

20 watermark data, but includes an API that can be called by the 
Capture program to decode a watermark from the data then 
being received. The API returns the decoded watermark data 
to the Capture program, which relays same to the clearing- 
house, as above. In still another embodiment, the watermark 

25 decoder fonns part of the Capture program, which both 
decodes the watermark and relays it to the clearinghouse 
when the Capture program is invoked by the user. 

There are various techniques by which the Capture pro- 
gram can be selectively invoked. One is by a keyboard macro 

30 (e.g. by a combination of keyboard keys). Another is by a 
program icon that is always presented on the screen , and can 
be double-clicked to activate. (Again, confirmation processes 
may be called for, depending on the likelihood of inadvertent 
invocation.) Many other techniques are likewise possible. 

35 In the just-contemplated scenario, the Capture operation is 
invoked while the user is downloading music from a non- 
commercial web site. This seems somewhat redundant, since 
the downloading — itself— is transferring music to the user's 
computer However, the Capture operation provides added 

40 value, 

hi the case of streaming audio, the audio is not typically 
stored in a location in which it can be re- used by the con- 
sumer. It can be listened-to as delivered, but is then gone. 

45 Capturing the audio provides the user a copy that can be 
played repeatedly. 

In the case of downloaded music files, the music may have 
been encoded to prevent its recordal on other devices. Thus, 
while the user may download the music oh to a desktop com- 

30 puter, copy- prevent ion mechanisms may prevent use of that 
file anywhere else, e.g. on a portable music appliance, Again, 
Capturing the audio provides the user a copy that can be 
transferred to another device. (The music file provided by the 
clearinghouse can have copy-prevention limits of its own — 

55 e.g., the file can be copied, but only once, or the file can be 
copied only onto devices owned by the user.) 

(Confirmation of device ownership can be implemented in 
various ways . One is to identify to the clearinghouse all music 
devices owned by a user at the time the user registers with the 

60 clearinghouse (supplemented as necessary by later equip- 
ment acquisitions). Device IDs associated with a user can be 
stored in a database at the clearinghouse, and these can be 
encoded into the downloaded music as permitted devices to 
which the file can be copied* or on which it can be played.) 

65 The commerce opportunity presented by non- commercial 
music web- sites is but one enabled by digital watermarks. 
There are many others. 
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To take one example, consider the media by which music 
and artists are presently promoted. In addition to radio air- 
time, these include music videos (a la MTV), fan magazines, 
web advertisements, graphical icons (e.g. the Grateful Dead 
dancing bears), posters, live events, movies, etc. Water- s 
marked data can be used in all such media as a link in a 
commercial transaction. 

A poster* for example, typically includes a photo of the 
artist, and may comprise cover-art from a CD. The photo/art 
can be digitally watermarked with various types o f data .e.g., i o 
the artist's identify, the record label that distributes the artist 1 s 
work, the music project being particularly promoted by the 
poster (e.g. a CD, or a concert tour), a fan web- site related to 
the artist, a web- site hosted by the record label for selling 
audio in CD or electronic form, a web-site from which free is 
music by the artist can be downloaded, data identifying the 
poster itself, etc. 

A user, equipped with a portable appliance that merges the 
functions of palmtop computer and digital camera, can snap 
an image of the poster. The processor can decode the water- 20 
marked data, and initiate any of various links based on the 
decoded data. 

In au exemplary embodiment, after snapping the picture s 
the user invokes a software program on the device that 
exposes the various links gleaned from the snapped image 25 
data. Such a program can, for example, present the option of 
linking to the artist's fan web site, or downloading free 
streaming audio or music clips, or ordering the promoted 
CD), or requesting the above- noted clearinghouse to down- 
load a personal copy of selected song(s) by the artist to the 30 
user's personal music library, etc. (The device is presumed to 
have a wireless internet link. In devices not having this capa- 
bility, the requested actions can be queued and automatically 
executed when a link to the internet is available.) 

Still more complex transactions can be realized with the 33 
use of a remote database indexed by digital watermark fields 
decoded from the poster. For example, the poster may pro- 
mote a concert tour. Fields of the digital watermark may 
identify the artist (by a code or full text), and a web site or IP 
address. The user appliance establishes a Link to the specified 40 
site, and provides the artist identifier. In response, the site 
downloads the tour schedule for that artist, for display ou me 
device. Additionally, the downloaded/displayed information 
can include a telephone number that can be used to order 
tickets or, more directly, can indicate the class of seats still 45 
available at each (or a selected) venue, and solicit a ticket 
order from the user over the device. The user can supply 
requested information (e.g. mailing address and charge card 
number) over the return channel link (wireless or wired, as the 
case may be), and the ticket(s) will be dispatched to the user. 50 
In the case of a wireless link, all of this can occur while the 
user is standing in front of the movie poster 

Siiiiilar systems can be implemented based on watermark 
data encoded in any other promotional media. Consider 
music videos. Using known TV/computer appliances, water- 55 
mark data added to such videos can readily be decoded, and 
used to establish links to audio download, CD- sales, fan club, 
concert ticket outlet web sites, etc., as above. 

Even live events offer such watermark-based opportuni- 
ties. The analog audio fed to public address or concert speak- 60 
ers can be watermarked (typically before amplification) to 
encode plural-bit digital data therein. A next generation per- 
sonal music appliance (e.g. one with a wireless interface to 
the internet) can include analog record capability (e.g. a built- 
in microphone, analog-to-digital converter, MP3 encoder, 65 
coupled to the unit's semiconductor memory). A user who 
attends a live event may record an excerpt of the music. The 


watermark can then be decoded, and the extracted data used to 
access the links and commerce opportunities reviewed above. 

Cinema movies offer both audio and visual opportunities 
for watermark-based commerce opportunities. Either 
medium can be encoded to convey information of the types 
reviewed above. A personal appliance with image- or audio- 
capture capabilities can capture an excerpt of the audio or 
imagery, decode the w r atermark data therefrom, and perform 
any of the linking, etc., functions reviewed above. 

Ihe consumer- interest watermarks reviewed above are 
only exemplary. Many others will be recognized as useful. 
For example, promotional clips presented before a feature 
film presentation can include watermark data that point (ei- 
ther by a literally encoded web address link, or by an ID code 
that indexes a literal link in a remote link database) to 
reviewer critiques of the previewed movies. Watermark data 
in a featured film presentation can lead to web sites with 
information about the movie stars, the director, the producer, 
and can list other movies by each of these persons. Other 
watermark-conveyed web links can present opportunities to 
buy the movie on videotape, to purchase the movie 
soundtrack, to buy movie-related toys and games, etc. 

More on Device Control 

Much of the foregoing has focused on watermark encoding 
to provide enhanced customer experiences or opportunities. 
Naturally, watermarks data can alternatively, or additionally, 
serve the interests of the media owner. 

To illustrate, consider watermarked music. The media 
owner would be best served if the watermark serves dual 
purposes: permissive and restrictive. Pennissively, music 
appliances can be designed to play (or record) only music that 
includes an embedded watermark signaling that such activity 
is authorized. By this arrangement, if music is obtained from 
an unauthorized source and does not include the necessary 
watermark, the appliance win recognize that it does not have 
permission to use the music, so will refuse requests to play (or 
record). 

As noted, music appliances can respond restrictively to the 
embedded watermark data to set limits on use of the music. 
Fields in the watermark can specify any or alJ of (or others in 
addition to) (a) the types of devices on which the music can be 
played (b) the types of devices on which the music can be 
recorded, (c) the number of times the music can be played; (d) 
the number of times the music can be recorded, etc. 

The device restrictions (a ) and (b) can be of various types. 
In some embodiments, the restrictions can identify particular 
units (e.g. by serial number, registered owner, etc.) that are 
authorized to play/record the encoded music. Or the restric- 
tions can identify particular classes of units (e.g., battery- 
powered portable players with music memories of less than 
50 megabytes, disk- based dedicated music appliances, gen- 
eral purpose personal computers, etc.) Or the restrictions can 
identify particular performance quality criteria (e.g., two 
channel, 16-bit audio at 44.1 KHz sample rate, or lower 
quality). 

The use restrictions (c) and (d) can likewise be of various 
types. Examples include "do not copy," "copy once only" 
"unrestricted copying permitted," "play once," "play N 
times" (where N is a parameter specified elsewhere in the 
watermarked data, or by reference to a database indexed by a 
watermark data field), "unrestricted playing permitted, 17 etc. 

It is straightforward to design a music appliance to respond 
to usage limits of zero (e.g. "do not copy") and infinity (e.g. 
"unrestricted copying permitted" and "unrestricted playing 
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permitted**) - The device simply examines one or more bits in 
the watermark data, and permits (or refuses) an operation 
based on the value thereof. 

Implementation of the other usage-control restrictions can 
proceed in various ways. Generally speaking, the stored 5 
music ca n be altered to give e (Feet to the usage-control restric- 
tions. For example* if the music is "record-once," then at the 
time of recording, the appliance can alter the music in a 
fashion indicating that it now has "do not record 11 status. This 
alteration can be done, e.g r , by changing the watermark data 1 0 
embedded in the stored music (or adding watermark data), by 
changing other data stored in association with the music, etc. 
If the original signal is stored (as opposed, e.g., to a streaming 
signal, such as an internet or wireless transmission), it too 
should be so- altered. ] 5 

Likewise with playback limitations. The number of play- 
backs remaining can, e.g., he encoded in an updated water- 
mark in the music, be tracked in a separate counter, etc. 

More particularly considering the "copy once" usage 
restriction, an illustrative embodiment provides two distinct 20 
watermark payload hits : n "copy once" bit and a "copy never" 
bit. When originally distributed (whether by internet, wire- 
less, or otherwise), the "copy once 1 ' bit is set, and the "copy 
never" bit is un-set. 

When music encoded in this fashion is provided to a com- 25 
pliant recording device, the device is audiorizedto make one 
copy, (A compliant device is one that, recognizes encoded 
watermark data, and behaves as dictated by the watermark.) 
When this privilege is exercised, the recording device must 
alter the data to ensure that no further copying is possible. In 30 
the illustrated embodiment, tins alteration is effected by the 
recording device adding a second watermark to both the 
music, with die "copy never" bit asserted. The second water- 
mark must generally be encoded in an "orthogonal" domain, 
so that it will be detectable notwithstanding the continued 35 
presence of the original watermark. Compliant equipment 
must then check for both watermarks, and refuse to copy if 
either is found to have the "copy never" bit asserted. 

One advantage to this arrangement is lhat if the watermark 
s igna 1 has undergone some form of corruption ( e.g . scaling or 40 
resampling), the first watermark may have been weakened In 
contrast, the second watermark will be native to the corrupted 
signal, and thus be more easily detected. (The corruption may 
also contribute to the orthogonality of one watermark relative 
to the other, since the two watenn arks may not have precisely j 5 
the same time base or other foundation.) 

An alternative approach is not to encode the "copy never" 
bit in the original music, but leave this bit (in whatever mani- 
festation) blank (i.e. neither 'T'nor^O"). hi transform-based 
watermark techniques , this can mean leaving transform coef- 50 
ficient(s) corresponding to the "copy never" bit un-changed. 
If the watermarking is effected hi the temporal sample 
domain (or spatial domain, for image data), this can mean 
leaving certain samples (pixels) unmodified. The recording 
device can then alter the transform coefficients and/or 55 
samples as necessary to assert the previous ly-uuencoded 
"copy never' 1 bit when the permitted recording is made. 

hi such a system, compliant recording devices check for 
the "copy never" bit in the sole watermark, and refuse to make 
a copy if it is asserted (ignoring the value of any "copy once" 60 
bit). 

A third approach to "copy once 51 is to set both the "copy 
once" and "copy never'' bits, but set the former bit very 
weakly (e.g. using lower gain and/or high frequency OCT 
coefficients that do not survive certain processing). The frail 05 
"copy once" bit is designed not to survive common corrup- 
tions, e.g., resampling scaling, digital to analog conversion, 
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etc. To liirther assure that the "copy once" bit is lost, the 
recording device can deliberately add a weak noise signal that 
masks this bit (e.g. by adding a noise signal in the frequency 
band whose DCT coefficient conveys the "copy once" bit). In 
contrast, the "never copy" bit is unchanged and reliably 
detectable. 

In such a system, compliant devices check for die "copy 
once" bit in the sole watermark, and refuse to make a copy if 
it is not detected as set. 

These three examples are but illustrations of many possible 
techniques for changing the rights associated with a work. 
Many other techniques are known. See, e.g, , the proposals for 
watennark-based copy control systems tor digital video at the 
Copy Protection Technical Working Group, http://www.dvc- 
c.couVdhsg/, from which certain of the foregoing examples 
are drawn. See also Bloom et al, "Copy Protection for DVD 
Video," IEEE Proceedings, Special Issue on Identification 
and Protection of Multimedia Information, June, 1999. 

S capability 

One feature that is desirable in many detector embodi- 
ments is scaleability. Tliis refers to die ability of a detector to 
scale its computational demands to match the computational 
resources available to it. If a detector is running on a high 
performance Pentium m workstation, it should be "doing 
more" than if the same detector is running on a slow m lcro- 
controller. One way scalability can be achieved is by process- 
ing more or less chunks of input data (e.g. temporal excerpts 
of music, orblocks/macroblocks of pixels in a frame of video 
data) to decode watermarks. For example, an input audio 
stream might be broken into chunks of one second each , A fast 
processor may complete decoding of each chunk in less than 
a second, permitting it successively to process each chunk in 
the data stream, In contrast, a slow processor may require two 
and a half seconds to decode the watermark from a chunk. 
While it is processing a first chunk, the second and third pass 
by un-decoded. The processor next grabs and processes the 
fourth chunk, permitting the fifth and sixth to pass by un- 
encoded. 

The detector running on the last processor is clearly more 
difficult to "fool " and yields a decoded watermark of higher 
confidence. But both systems decode the watermark, and both 
operate in "real time.' 1 

The skipping of input data in the temporal (e.g. music or 
video) or spatial (e.g. image or video) domain is but one 
example of how scaleability can be achieved. Many other 
approaches are known to those skilled in the art. Some of 
these alternatives rely on spending more or less time in the 
data analysis phases of watermark decoding, such as cross- 
correlation operations. 

Reference has been made to watermarked UTDs as refer- 
ring to a database from which larger data strings (e.g. web 
addresses, musician names, etc.) can be retrieved. In some 
embodiments, the data record referenced by a UID can, in 
turn, point to several other database records. By such arrange- 
ments, it is often possible to reduce the payload of the water- 
mark, since a single UID reference can lead to several differ- 
ent data records. 

Production Tools 

In the prior art, the watermark embedded in a source mate- 
rial is typically consistent and static through a work- un- 
clianging from beginning to end. But as will be recognized 
from the foregoing, there are many applications that are better 
served by changing the watermark data dynamically during 
the course of the work. According to another aspect of the 
invention, a production tool is provided that facilitates the 
selection and embedding of dynamically-changing water- 
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mark data. One such embodiment is a software program hav- 
ing a user interface that graplneally displays the different 
watermark fields that are being embedded in a work, and 
presents a library of data (fexfually or by icons) that can be 
inserted into each field, and/or permits the user to type in data 
to be encoded. Another control on the Uf controls the advance 
and rewind of the media, permitting the user to determine the 
location at which different watermark data begins and ends. 
Graphical paradigms known from video- and audio-editing 
tools can be used to indicate the starting and ending frames/ 
samples for each different watermark payload. 

Such a tool can be of the standalone variety, or can be 
integrated into the desktop audio- and video- production and 
editing tools offered by vendors such as Avid, Adobe, Jaleo, 
Pinnacle Systems, Soun&Forge, Sonic Foundry, Xing Tech- 
nology, Prosoniq, and Sonic Desktop Software, 

Payment-Based Systems 

Another aspect of the present invention is the use of anony- 
mous payment tokens that can be used to obtain content on the 
web. In one embodiment, a token comprises a 1 28-bit pseudo- 
random number, to which additional bits identifying an issu- 
ing bank (or other issuing institution) are appended. (The 
additional bits can be the IP address of a web server of the 
bank, a routing number identifying the bank for electronic 
wire transfers, or other identifier.) The 128-bit numbers are 
randomly generated by the bank — commonly as needed — 
and each represents a fixed increment of money, e,g. tencents. 

A consumer wishing to have a store o f currency for such 
commerce pays the bank, e.g., $10 in exchange for 100 
tokens. These tokens are transferred electronically to disk or 
other storage in the consumer's computer in response, e.g., to 
a credit card authorization, or may be provided by diskette or 
other storage medium over the counter at a bank branch (in 
which case the consumer thereafter copies the numbers into 
storage of Ins or her computer). (Outlets other than banks can 
of course be employed for distributing such numbers , much in 
the manner that convenience and many grocery stores com- 
monly issue money orders.) 

Imagine that the consumer wishes to view the final quarter 
of a Trailblazer basketball game mat aired on television a 
week ago. (The consumer may have either missed the game, 
or may have seen it but wants to see the last quarter again.) 
Ihe user directs a web browser to a web site maintained for 
such purpose and performs a search to identity the desired 
program. (Typically, the web site is maintained by the propri- 
etor that holds the copyright- in the material, but this need not 
be the case. Some material may be available at several web 
sites, e.g., maintained by ABC Sports, the National Basket- 
ball Association, and Sports Illustrated.) The search can use 
any of various known search engines, e.g., Jhfoseek. Verity, 
etc., and can permit searching by title terms, keywords, date 
of airing, copyright owner, etc. By typing in, e.g., the keyword 
'Trailb lasers 1 and the date 'Apr. 26, 1999,' the consumer is 
presented a listing of videos available for download. One, 
hopefully, is the requested game. With each listing is an 
indication of an associated nominal charge (e.g. SO cents). 

On clicking on a hypertext link associated with the desired 
basketball game, the viewer is presented a further screen with 
one or more options . 'Ihe first ofthe listed options is die entire 
game, with commercials. The charge is the nominal charge 
presented on the earlier screen (i.e, 80 cents). Other options 
may include the first, second, third, and fourth quarters ofthe 
game individually, each of which — save the last, costs 20 
cents. The last may be charged at a premium rate, e.g., 30 
cents. Clicking on the desired video option yields a further 
screen through winch payment is effected. 
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To pay for the requested video, the consumer instructs his 
or her computer to transfer three of the earlier-purchased 
tokens over the web to the video provider. Various user inter- 
lace metaphors can be employed to facilitate this transfer, 

5 e.g., permitting the user to type the amount of money to be 
transferred in a dialog box presented on-screen, or dropping/ 
dragging icons representing tokens from an on- screen "wal- 
let" to an on-screen 'ticket booth'* (or over an icon or thumb- 
nail representing the desired content), clicking on an 

i o "increment' * counter displayed adjacent the listing ofthe con- 
tent, eta Once me consumer has authorized a transfer of 
sufficient tokens, the consumer's computer sends to the web 
site (or to such other web address as HTML encoding in the 
viewed web page may indicate) the tokens. This transmission 

1 5 simply takes the form of the three 1 28+ bit numbers (the l + 3 
inch eating the bank identifier) — in whatever packet or other 
format may be used by the internet link. Once dispatched in 
thi?; manner, the tokens are deleted from the user's computer, 
or simply marked as spent, (Of course, in other embodiments, 

2(3 a record ofthe expenditure may be stored in the consumer's 
computer, e.g., with the token contents and a record ofthe 
audio or video purchase to which they were applied.) 

Since the amount of money is nonunal, no encryption is 
provided hi this embodiment, although encryption can natu- 

25 rally be provided in oilier embodiments (e.g,, either in send- 
ing the tokens from the user to the web site, or earlier, in 
sending the tokens to the user). As will be seen, provided that 
the media provider immediately sends the tokens to the bank 
in real time, encryption is a nice feature but not mandatory 

io On receipt of the token data, the web site immediately 
routes the token data to the identified bank, together with an 
identifier ofthe media provider or account to which the funds 
represented thereby are to be credited. The bank checks 
w r hether the 138-bit numbers have been issued by that bank, 

35 and whether they have already been spent. If the numbers are 
valid, the bank updates its disk- based records to indicate that 
the three tokens have been spent and that the bank now owes 
the media supplier 30 cents, which it may either pay imme- 
diately (e.g., by crediting to an account identified by the 

ao media provider) or as one lump sum at the end ofthe month. 
The bank then sends a message to the web site confirming that 
the tokens were valid and credited to the requested account 
(Optionally, a message can be sent to the purchaser ofthe 
tokens (if known), reporting that the tokens have been 

45 redeemed.) 

hi response, the web site begins delivery ofthe requested 
video to the consumer, hi the illustrated embodiment, the 
video is watermarked prior to delivery, but otherwise sent in 
unencrypted fashion, typically in streaming format, but 

5Q optionally in file format. (Encryption can be used in other 
embodiments,) The watermarking in the illustrated embodi- 
ment is accomplished on-the- fly andean include various data, 
including the date of downloading, the download site, the 
destination IP address, the identity of the purchaser (if 

55 known), etc. 

The large size ofthe video and the small charge assessed 
therefor provide disincentives for the consumer making illicit 
copies. (Especially as to archival material whose value decays 
with time, there is not much after-market demand that could 

60 be served by illicit copies, making third party compilation of 
such material forre-distributiontinancially unattractive. First 
run video, and material that keeps a high value over time, 
would not be as well suited for such distribution, and could 
better employ technology disclosed elsewhere herein.) 

r^s In some embodiments, the integrity o f the received video is 
checked on receipt. This feature is described below in the 
section entitled Watermark- Based Receipts. 
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111 the illustrative system, nothing in the tokens indicates 
the identity of the purchaser. The web site knows the IP 
address of the site to which video was delivered, but need not 
otherwise know the identity of the purchaser. The bank would 
probably maintain a record of who purchased the tokens, but 5 
need uot. In any event, such tokens could thereafter be 
exchanged among consumers, resulting in anonymity from 
the bank, if desired. 

As described above, the video excerpts from which the 
consumer can select include commercials. At some sites, to 
video may be provided from which the commercials have 
been excised, or which is delivered in a manner that skips past 
the commercials without transmitting same to the consumer. 
Such video will naturally command a premium price. In some 
embodiments, the difference in price is electronically cred- 1 5 
ited as compensation to accounts maintained for {or by) the 
advertisers, whose advertisements are not being viewed by 
such consumers. (The identification of advertisers to be cred- 
ited is desirably permanently encoded in the video, either 
throughout the video (if the video has had the commercials 20 
removed therefrom), or by data in the commercials them- 
selves (which commercials are skipped for transmission to 
the consumer, but can still be decoded at the video head-end. 
Such encoding can be by in-band watermarking or other- 
wise.) 25 

While the foregoing discussion particularly considered 
video as the desired content, the same principles are equally 
applicable in connection with audio, still imagery, and other 
content. 

The token-based payment method is but one of many that ™ 
can be employed; the literature relating to on-line payment 
mechanisms is extensive, and all such systems can generally 
be here-employed. 

Tracking 1 28-bit tokens can be a logistical problem for the 
bank. One appro ach is to have a memory with 1 0 1 28 locations, 35 
and at each location store a two-bit value (e.g. 00=never 
issued; Ol^issued but not spent; 10-issued and spent; 
Unreserved). More complete data could alternatively be 
stored, but such a memory would be unpractically large. 

One alternative approach is to hash each 128-bit number, ao 
when issued, to a much smaller key value (e.g. 20 bits). A 
memory with 1 0 20 locations can be indexed by this key. Each 
such location can include four data: an issued 1 28-bit token 
number that hashes to that value, first and second date fields 
indicating the date/time on which that token was issued and 45 
redeemed, respectively, and a link specifying the address of a 
next, memory location. That next memory location (outside of 
the original 1 0 20 locations) can include four more data, this 
time for a second issued- 1 28 -bit token number thai hashed to 
the original key value, two date fields , and again with a link to 50 
a subsequent storage location, etc. 

When a 1 2 8 -bit random number is generated, the original 
memory location indexed by the hash code of that number is 
checked for an earlier number of the identical value (to avoid 
issuance of duplicate tokens). Each successive location in the 35 
linked chain of memory locations is checked for the same 
1 28-bit number. When the end of the linked chain is reached, 
the bank knows that the 128-bit random number has not 
previously been issued, and writes that number in the last- 
addressed location, together with the date of issuance, and a 60 
link to a next storage location. 

When a 128-bit token is received, the same liuked-hst 
processing occurs to identify a first location, and to thereafter 
step through each subsequent location until a match is found 
between the token number and the number stored in one of the 65 
linked memory locations. When found, that number is 
marked as redeemed by writing a redemption date/time in the 


corresponding field. If the search reaches the end ofthe linked 
chain without finding a match between the stored numbers 
and the token number, the token is treated as invalid (i.e. not 
issued by that bank). 

Other manners of tracking the large number of possible 
token numbers can of course be used; the foregoing is just 
exemplary. Or the tokens needn't be tracked at all. Such an 
arrangement is highly practical if the token has sufficient bits. 
With the illustrated 128 bits, for example, the chance of two 
identical tokens being issued is irimiitesitnally small so 
checking for duplicate issuance can be omitted if desired. In 
such case, the bank can simply maintain an ordered list ofthe 
token numbers still outstanding and valid. As new tokens are 
dispensed* their token numbers are added to the list. As tokens 
are redeemed, their numbers are deleted from the list. Known 
list processing techniques can be employed to speed such 
search, update, and detete actions, 

Watermark-Based Receipts 

Pay-for-content applications commonly assume that if 
content is transmitted from a server (or head-end, etc.), it is 
necessarily received. Sometimes tins assumption is wrong. 
Network outages and interruptions and internet traffic load 
can diminish (e.g., dropped video frames), or even negate 
(e.g., failed delivery), expected consumer enjoyment of con- 
tent. In such cases, the consumer is left to haggle with the 
content provider in order to obtain an adjustment, or refund, 
of assessed charges. 

Watermarks provide a mechanism lor confirming receipt of 
content. If a watermark is detected continuously during a 
download or other delivery event, a software program (or 
hardware device) can issue an electronic receipt attesting that 
the content was properly delivered. This receipt can be stored, 
and/or sent to the content distributor to confirm delivery. 

In one embodiment, a content receiving device (e.g., com- 
puter, television or set-top box, audio appliance, etc.) peri- 
odically decodes a watermark from the received content to 
confirm its continued reception. For example, every five sec- 
onds a watermark detector can decode the watermark and 
make a record ofthe decoded data (or simply record the fact 
of continued detection of the same watermark). When a 
changed watermark is detected (i.e., reception of a different 
content object begins), the duration of the previously-re- 
ceived content is logged, and a receipt is issued. 

In a related embodiment* the last portion (e.g,, 5 seconds, 
frame, etc.) of the content bears a different "end of content" 
watermark that triggers issuance of a receipt. Such a water- 
mark can indicate the length of the content, to serve as a 
cross-check against the periodic watermark polling. (E.g., if 
periodic sampling at 2 second intervals yields 545 samples 
corresponding to the same content, and if the "end of content" 
watermark indicates that the content was 1090 seconds long, 
1hcn receipt ofthe entire content can be confirmed.) 

hi another embodiment, the watermark can change during 
the course of the content by including, e.g., a datum that 
increments every frame or other increment of time (e.g., 
frame number, time stamp, etc.). A watermark detector cart 
monitor the continued incrementing 0 f this datum throughout 
the content to confirm that no part was garbled (which would 
destroy the watermark) or was otherwise missing. Again, at 
the end of delivery, the receiving system can issue a confir- 
mation that XXX frames/seconds/etc. of the identified con- 
tent were received. 

One application of such teclmology is to bill for content 
based on receipt, rather than transmission. Moreover, billings 
can be adjusted based on percentage of content-value 
received. If delivery is interrupted mid- way through (e.g., by 
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the consumer disabling the content-receiving device), the 
nominal billing for the content can be halved. Some pro- 
longed content, e.g.* tefevised/web-broadcast university 
classes, cannot be "consumed 1 * in one session, and arc thus 
particularly well suited for such pay-as- you-consume billing. 

Another application of such technology is hi advertising 
verification. Presently, ads are tracked by transmission or, less 
frequently, by detection of an embedded code on receipt (cf., 
Nielsen Media Research's Pat. Nos, 5,850,249 and 5,737, 
025)- However, such reception-detectors — once triggered — 
generally do not furthernote the length of time that the adver- 
tising was received, so the same data is produced regardless of 
whether only five or fifty seconds of a commercial is pre- 
sented. Watermark monitoring as contemplated herein allows 
the duration of the advertising impression to be precisely 
tracked. 

In one application of this technology, recipients of adver- 
tising are provided incentives for viewing advertising in its 
entirety. For example, a content-receiving device can include 
a watermark detector that issues a receipt for each advertise- 
ment that is heard/viewed in its entirety. These receipts can be 
redeemed, e.g., for content tokens as described elsewhere 
herein, for monetary value, etc. In some embodiments, 
receipts are generic and can all be applied to a desired pre- 
mium, regardless of the advertisements through which they 
were earned. In other embodiments, the receipts are associ- 
ated with the particular advertisers (or class of advertisers). 
Thus, a TV viewer who accumulates 50 receipts from adver- 
tising originating from Procter & Gamble may be able to 
redeem same for a coupon good for $2,50 off any Procter & 
Gamble product, or receipts from Delta Airlines may be 
redeemed for Delta frequency flier miles (e.g., at a rate of one 
mile per minute of advertising). Such incentives are particu- 
larly useful in new forms of media that give the consumer 
enhanced opportunities to fast- forward or otherwise skip 
advertising, 

(Although the foregoing "receipt" concept has been 
described in conjunction with watermark data (and use of 
watermark technology is believed to be inherently advanta- 
geous in this application), the same principles can likewise be 
implemented with ancillary data conveyed by other means.) 

Master Global Address 

As suggested above, it is desirable that each piece of con- 
tent have a web address (the "Master Global Address" 
(MGA), or "Master IP Address") associated with it. Such 
address is typically conveyed with the content, e.g., by an IP 
address watermarked therein. 

Consider a consumer who downloads a streaming video 
having m JEnghsh language soundtrack. The viewer may not 
speak English, or may otherwise prefer to listen to the 
soundtrack in another language. The user can decode the 
watermark data embedded in the video and initiate a link to 
the associated web address. There the user is presented with a 
list of soundtracks for that content object in other languages. 
The viewer can click on the desired language and receive 
same via a second simultaneous transmission (e.g, T a second 
socket, channel). The consumer's audio/Video appliance can , 
substitute the desired audio track for the default English track. 

If the streaming video and the alternative soundtrack are 
hosted on the same server, synchronization is straightfor- 
ward. The process governing transmission of die alternative 
soundtrack identifies the process that is streaming video to the 
same IP address. Based on SMPTE, or other time/frame data, 
the former process syncs to the tatter. (If the two data streams 


don't originate through the same server, time/frame data can 
be relayed as necessary to the alternative soundtrack server to 
effect synchronization.) 

Another application of the Master GJobal Address is to 
serve as a point to which monitoring stations can report the 
presence, or passage, of content. Consider, for example, a 
copyright-aware node through which content signals pass, 
e.g., a computer node on a network, a satellite transponder, 
etc. Wheuever the node delects passage of a media object 
(e.g., by reference to a file extension, such as MP3, JPG, AVI, 
etc.), it sends a "ping" over the internet to The address encoded 
in the object, simply reporting passage of the object. Similar 
monitoring facilities can be provided in end user computers, 
e.g., reporting RleOpen, FileSave, Printing, or other use of 
content bearing MGA data. 

This system can be expanded to include "ping >s and "pong" 
phases of operation. When a software application (or a user 
appliance, such as a video or audio playback device) encoun- 
ters a media object (e.g., at time of file open, at time of 
playback, etc), it pings the MGA site to report the encounter. 
The MGA site "pongs" back, responding with instructions 
appropriate lo the encounter. For example, if the object 
requires payment of a fee before full functionality or access is 
to be granted, the MGA site can respond to the application 
with instructions that the object be used (e.g., played back) 
only in some crippled state preventing the user's full enjoy- 
ment (e.g., impaired resolution, or impaired sound quality, or 
excerpts only, etc.). The MGA site can also inform the user 
application of the terms (e.g,, payment) by which full func- 
tionality can be obtained. The application can graphically or 
audibly present such information to the user, who can autho- 
rize a payment, if desired, so that the content can be enjoyed 
in a iess- (or un-) crippled state. On receipt of the payment 
authorization^ the MGA site can inform the user application 
that enhanced access/usage rights have been purchased, and 
that the application may proceed accordingly. 

Yet another applicauonofthe MGA is to present the user of 
a content object a menu of options tliat is customized to that 
obj ect. 

hi current graphical operating systems, when a user clicks 
on an icon (e.g., with the right mouse button), a menu is 
presented detailing actions that can be undertaken in connec- 
tion with the icon, or the file represented thereby. Such 
options are pre-programmed (i.e., static), and are typically 
determined by the operating system based solely on the file 
extension. 

In accordance w r ith this aspect of the present invention, 
clicking on an icon representing a media object initiates an 
internet link to the MGA site associated with the object. The 
MGA site responds with data that is used to customize the 
menu of options presented to the user in connection with that 
particular object. 

Consider an icon representing a IPG image file. Right- 
clicking on tlie icon may yield a menu that gives the user 
various options presented by the operating system (e.g,, 
delete, compress, rename), and additional options custom- 
ized in accordance with dau from the object's MGA site. 
These customized options may include, e.g., 

(a) open in 1 00x1 50 pixel format for free; 

(b) open in 480x640 pixel format for ten cents; 

(e) open in 960x1280 pixel format for twenty cents; 

(d) purchase rights to use this image in a newsletter having 
a circulation of under 1000 for SI .25; 

(e) display a complete listing of license options. 

Clicking on options (b) or (c) initiates a commerce appli- 
cation through which funds are electronically transferred to 
the MGA site (by the above-described tokens or otherwise). 
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In response, the MGA site responds (e.g., with TCP/IP or 
HTML instructions) authorizing an application on the user's 
computer to open the file in the requested manner. (The 
default application for JPG applications can then automati- 
cally be launched, or the computer may first query the user 5 
whether another application should be used instead.) 

Clicking on option (d) proceeds as above, and permits full 
use of the image on the computer. Moreover, the MGA site 
sends a digital certificate to the user's computer memorializ- 
ing the usage rights purchased by the consumer. 10 

In this particular arrangement, no access control is placed 
on the content, e.g., by encryption, secure container technol- 
ogy, or the like. The nominal fees, and the ease of licensing, 
make it simple for the user to "do the right thing 11 and avoid 
copyright liability. In other embodiments, of course, known 1 5 
access control techniques can be used to limit use of the object 
until the requisite payment has been made. 

Naturally, recordsof all such transactions are also loggedat 
the MGA site. 

Clicking on option (e) opens a browser window on the 20 
user's computer to a web site that presents a complete listing 
of license options available for that image. (The address of 
mis web site is included in customization data relayed to the 
user device from the MGA site, but. uot explicitly shown to the 
user on the menu. ) Through such web site, the user can select 25 
desired rights, effect payment, and receive the necessary 
authorization for software applications ou the user's com- 
puter (or other media appliance) to open and/or process the 
content. 

The object on which the user "clicks" needn't be an icon. It 30 
can be an image or other graphical representation. (And a 
"click" isn't necessary; a voice command or otber signal may 
be used to the same effect with an audio clip or selection.) 

Consider the popular merchandising of books and CDs 
over the internet. A JPG or other image file depicting the cover 35 
of a book, or the artwork of a CD cover, can be treated as a 
media object, and can include a watermarked MGA pointer. 
Right- clicking on such an image of a book cover could, 
through the MGA site, present to the user a menu of options 
that includes — in addition to those normally presented in ao 
conjunction with a JPG file — the following: 

(a) "See the review of this book published in the New York 
Times on Apr. 19, 1999 11 

(b) "See the list of reviews of this book at Amazon, com" 

(c) "Enter your own review of this book, for posting on 45 
Amazon .corn" 

(d) "See today's sales rank of this book at Amazon, com" 

(e) "Purchase this book trom Amazon.com for SI 6.95" 

(f) "Purchase this book from Barnesandnob3e.com for 

S 19.95 and receive a $5.00 credit towards your next 50 
purchase" 

(g) "Link to the web site that tells about the release of this 
title as a motion picture (presently scheduled to open on 
Oct. 10, 1999)" 

(h) "Link to the Yahoo listing of web sites relating to this 55 
book" 

(i) "Search Lycos for listings relating to this book." 

If die user selects one of the purchase options from the 
menu, a pre- stored e-conuuerce profile containing the user 
name, credit card number, billing address, ship-to address, 00 
etc, possibly in the form of an encrypted object - could be 
sent to the MGA site (or to the bookseller) to effect the 
purchase, or such selection could initiate display of additional 
screens or sub-menus through which the user would manually 
enter or select such information for transmission. 65 

Others of the selections cause a new browser window to 
open ou the user's computer, opening to a URL specified in 


data relayed from the MGA site but not displayed to the user 
in the menu. Appropriate HTML instructions can be gener- 
ated to effect a particular query or other operation at the 
specified URL. 

In some embodiments, the customized menu presents only 
a single choice in addition to those normally provided by the 
operating system, e.g., "Link to home." Clicking on this 
option opens a browser window to a home page at the MGA 
for that object. On that page, the user is presented with all of 
the ibregoing options, and more (possibly including advertis- 
ing graphics or multi-media). Such objects can serve as pow- 
erful marketing agents. Returning to the example discussed 
above, a JPG image file of a book cover may have, as its 
MGA, a web page hosted by a particular bookseller, provid- 
ing purchase options and other information for thai book. 
Marketing of books (or CDs, or cars, or consumer appliances, 
or virtually anything else) can be effected by disseminating 
such vendor* issued JTGs as widely as possible. Some book 
cover JPGs may be distributed by Amazon.com, others by 
Bames&Noble.com, others by Borders.com — each pointing 
back to a different MGA through which purch ase transactions 
for that book may be performed. 

Returning Lo the MGA- customized menus, these needn't 
be limited to menus resulting from clicking on an icon or 
image (or signaling during an audio excerpt). Drop-down 
menus in application programs can likewise be populated 
with customized options, in accordance with customization 
data obtained from the MGA site for die object presently 
being accessed or used. Most graphical operating systems and 
application programs have well developed toolsets permitting 
such menu customization. Again, other data relayed from the 
MGA site is not shown to the user, but is employed by the 
computer (e.g a browser program) to carry out menu options 
selected by the user. 

Again the foregoing techniques are equally applicable for 
still images, audio, video, and other forms of content, and can 
readily be adapted for use both with general purpose comput- 
ers, software applications, and specialized media appliances. 

While, for expository convenience, the foregoing discus- 
sion contemplated embedding a literal URL address in the 
object as the MGA, more typically this is not the case, 
hi stead, the MGA more commonly comprises identification 
data for the object (e.g. a ! 28-bit random ID), together with 
the URL for a name server computer that serves many (per- 
haps millions) of such objects (an example of the latter is the 
Digimarc MarcCentre server). 

To obtain the desired data as detailed above, the user's 
computer (sometimes termed a client computer) links to The 
name server computer and provides the ID of the object being 
processed. The name server computer uses this ED to query a 
databa se, and obtains from the database the current EP address 
to which such queries should be routed. The name server 
computer can relay the request from the client computer to the 
correct destination address, or can return the correct destina- 
tion address to the client computer, which can initiate such a 
link itself By such arrangement, the IP address ultimately 
associated with an object can be easily changed as needed, 
simply by changing the corresponding record in the name 
server database, without rendering obsolete legacy objects 
having out-of-date addresses encoded therein. 

In some embodiments, the URL of Hie name server needn't 
be included in the watennark. hi the absence of a specified 
URL, the client computer may direct such links to a default 
name server address instead (stored locally or remotely). If 
that server doesn't recognize the object ID, it can return an 
error code, or pass the query on to other name servers. Those 
servers, in turn, can pass the query along to still other name 
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set vers if they don't recognize the object ID. In this fashion, 
an exponentially- large number of name servers might be 
quickly polled for information relating to the identified 
object. Alternatively, rather than encoding the complete IP 
address of the name server in an object watermark, the first N 5 
(e.g., 16) bits of the object ID might be used as a short- liand 
for one of 65,536 predetermined name server addresses, in 
accordance with data stored locally (e.g., on RAM or disk in 
the user's computer) or remotely (e.g., at a default name 
server IP address). io 

While the basic concept idea behind embedding MGA data 
within an object is to point to a repository of data about the 
object, a pointer the other way may be achieved as well. 

As noted, the "ping' 1 application of MGA data permits an 
MGA site to be informed of sites through which its object t5 
passes. More general! y, the MGA site can log the originating 
address of each query it receives. Each such address can be 
presumed to have (or have had) a copy of the corresponding 
object. Media owners can thereby track the dissemination of 
copies of their media objects — at least insofar as use of such 20 
objects entails communicating with the associated MGA site. 

Such tracking o ffers a great number of opportunities, some 
in die area of commerce. The MGA site corresponding to the 
cover ait of a Garth Brooks CD, for example, can provide a 
listing of IP addresses of persons interested in that CD. Email 25 
or promotional data objects (e.g. , audio clips) can be sent to 
that list of addresses when a subsequent Garth Brooks CD is 
released. 

Such tracking also opens up a new dimension of internet 
searching. Presently, internet search engines use a brute force 30 
approach, visiting millions of pages across the web in order to 
identify* for example, a dozen instances of a given photograph 
f]Je. MGAs offer a shortcut to such brute force approaches. 
With the present technology, a search engine can And a single 
instance of a photograph file and, by detection of the MGA - 1S 
data watermarked therein, link to the corresponding MGA 
site. From the MGA site, the search engine can obtain a listing 
(if such queries are auUiorized) of some or all of the other sites 
known by the MGA site to have copies of that photograph file. 
(Providing such data to search engines is a commerce oppor- 40 
tunity for such MGA sites, which may permit such access to 
its listing of sites only in exchange for a fee. Or the MGA site 
may arrange to collect a tribute payment from the search 
engine proprietor each time the engine responds to a user 
query using data collected from the MGA site.) * s 

Many of the addresses logged by the MGA may not be 
publicly-accessible data stores. The search engine can check 
each listed address to ensure that the desired object is present 
and accessible before adding the address to its database. 

so 

Covert Tracing 

Co-pending application Ser. No. 09/1^5,380 describes 
ami-counterfeiting technology that, looks for the presence of 
digital data corresponding to bank note imagery in a com- 
puter system, and makes a covert, record of any attempt to 53 
process such data (e + g, f Scan, FileOpen, FileSave, Print, Kdit, 
etc.). Such records are hidden from the user of the system 
(using, e.g., various data encryption and obscuring tech- 
niques), but authorized law enforcement officials are pro- 
vided tools by winch these records can be recovered. The 60 
forensic data thereby obtained may prove useful in prosecut- 
ing counterfeiters. (Knowledge that a computer may be 
covertly storing evidence of attempted counterfeiting actions 
may prove as, or more, valuable in deterring counterfeiting 
man tli e c overt re cords them sel ve s . ) 65 

The same techniques can be employed to deter unautho- 
rized processing of audio, image, video, or content by media 


pirates. In one embodiment, a computer's operating system 
(including peripheral device drivers) monitors various data 
within the system (e.g., data sent to writeable storage media, 
or sent via a serial port or network connection, etc) for data 
bearing a do-not-eopy watermark. The presence of such data 
being sent, e.g., to a writeable disk or to a remote computer, 
indicates that the do-no t-eopy instruction has been circum- 
vented , hi such case, the operating system writes one or more 
covert records memorializing the activity, for possible use in 
criminal prosecution if the computer is lawfully seized. 

The example just-provided is but one of many monitoring 
and response techniques that may be employed to deter cir- 
cumvention of copy-protection or other access control sys- 
tems. Generally speaking, if content data is found where it 
shouldn't be, or is found used as it shouldn't be used, a 
corresponding record should be made. (Other intervention 
actions can be triggered as well; covert tracing is desirably 
just one of several parallel responses to suspected hacking.) 

Mcta-Data Accessed Using Watermarks 

Mela-data, in formats known as XML, SGML, and HTML, 
is widely used to communicate information about digital 
objects (e.g., author, keywords, price, rights, caption, etc.). 
More generally, meta-data can be thought of as any data 
construct which associates the name of a property (e.g., 
"author) , with the value 0 f the property (e.g. , "Mark Twain"). 
Such data commonly appears in a tag format, such as the 
following: 

<META y. U-JF- \7inhor" CONTENT- ' Mark 

Meta-data is commonly exchanged between server and 
client computers in conjunction with the digital objects to 
which they relate (e.g., the text of a Mark Twain book). 

As detailed herein, an important applicationofwatermark- 
ing is likewise to convey information about media in this 
case embedded within The media content, itself (e.g., provid- 
ing unique identification, establishing some basic behaviors 
such as do not copy, and providing links to extended func- 
tionality). 

For meta-data to be useful, it must be linked to associated 
content, whether in the context of a browser, application 
program, operating system, asset management system, search 
engine t etc. However, as detailed below, the content and the 
associated meta-tags needn't always be conveyed together. 

Consider an application program or other client process 
mat receives a watermarked media object. The watermark 
includes an MGA for that obj ect (which, as noted above, may 
not specify an ultimate IP address). Stored at the MGA site is 
meta-data corresponding to the object. By Jinking to the 
MGA site identified by the objects watemiark, the client 
computer can obtain the meta-data corresponding to the 
object. This data can be stored at the client computer and used 
just as any other meta-data, e.g., to define the local functions 
mat should be available for use with tliat object (e.g., buy, 
search, etc.) 

A particular example is an on-line catalog of stock photog- 
raphy. Each photograph is watermarked with MGA data. To 
identify the photographer, copyright date, price, telephone 
number, subject, etc, an application program can link to the 
MGA site for that photograph, and obtain the corresponding 
meta-data. This data can then be displayed or used as needed. 
Data objects of disparate formats thus can readily be handled 
within a single, simple application program, since the pro- 
gram needn't concern itself with the varying formats for the 
associated meta-data (a ssuming the name serv ers provide this 
data in standardized format). Substantial flexibility in pro- 
gramming and object formatting is thereby achieved. 
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Returning to the internet search engine example described 
above, MGAs may become recognized as repositories rich in 
meta-data for media objects. Specialized search engines may 
focus their data collection around such sites, and be able to 
quickly identify the MGA sites corresponding to various 
boolean combinations of meta-lag parameters. 

Asset Management/Containers 

Much has been written on the topic of asset rights manage- 
ment. Sample patent documents include I LS. Pat. Nos. 5,892, 
900, 5,715,403, 5,638,443, 5,634,012, 5,629,980 and laid- 
open European application EP 862,318. Much of the 
technical work is memorialized in journal articles, which, can 
be identified by searching for relevant company names and 
trademarks such as IBM's Crypto lope system, Portland Soft- 
ware's ZipLock system, the Rights Exchange service by Soft- 
bank Net Solutions, and the DigiBox system from Inter Trust 
Technologies. 

An exemplary asset management system makes content 
available (e.g. from a web server, or on a new computer's hard 
disk) in encrypted form. Associated with the encrypted con- 
tent is data identifying the content (e.g. a preview) and data 
specifying various rights associated with the content. If a user 
wants to make fuller use of the content, the user provides a 
charge authorization (e.g. a credit card) to the distributor, who 
then provides a decryption key, allowing access to the con- 
tent. (Such systems are often realized using object-based 
teclmology. In such systems, the content is commonly said to 
be distributed in a "secure container.") 

Desirably, die content should be marked (personalized/ 
serialized) so that any illicit use of the content (after decryp- 
tion) can be tracked. This marking can be performed with 
watermarking, which assures that the mark travels with the 
content wherever - and in whatever fonn - it may go. The 
watermarking can be effected by the distributor prior to dis- 
semination of the encrypted object— such as by encoding a 
UID that is associated in a database with that particular con- 
tamer. When access rights are granted to that, container, the 
database record can be updated to reflect the purchaser, the 
purchase date, the rights granted, etc. An alternative is to 
include a watermark encoder in the software tool used to 
access <e.g. decrypt) the content. Such an encoder can embed 
watermark data in the content as it is released from the secure 
container, before it is provided to the user. The embedded data 
can include a UID. This UID can be assigned by the distribu- 
tor poor to disseminating the container. Alternatively, the 45 
UID can be a data string not known or created until access 
rights have been granted. In addition to the UTD, the water- 
mark can include oilier data not known 1o the distributor, e.g. 
information specific to the time(s ) and manner(s) of accessing 
the content. 

As noted earlier, access rights systems can be realized with 
watermarks without containers etc. For example, in a trusting 
world, copyrighted works can be freely available on the web. 
If a user wishes to make lawful use of the work, the user can ^ 
decode its watermark to determine the work*s terms and 
conditions of use. This may entail linking to a web site speci- 
fied by the embedded watermark (directly, or through an 
mtermediate database), which specifies the desired informa- 
tion. The user can then arrange the necessary payment, and 
use me item knowing that the necessary rights have been 
secured. 

Remote Reconfiguration of Watermark Detectors 

In some cases, it is desirable to reconfigure watermark 
detectors remotely. Such functionality is desirable, for 65 
example, if a watermark system is hacked or otherwise com- 
promised. 


In accordance with this aspect of the present invention, 
some aspect of a watermark detector's operation is changed in 
response to a command. The change can take various forms. 
In watermark systems employing pseudo-random key data 
(e.g., spread spectrum spreading signals), the pseudo-random 
signal used for detection can be changed. In systems using 
DFT processing, the mapping between message bits and DFT 
coefficients can be changed, hi still other systems, the decod- 
ing can proceed as before, but the significance of one or more 
bits can be changed (e.g., bits that were normally interpreted 
as defining Field A can be interpreted as defining Field B, and 
vice versa). In yet other systems, the decoding can proceed as 
before, but the response of a device to a given watermark 
signal can be changed, fh still other systems, a set of software 
instructions can be re- written or re- ordered to effect a change 
m detector operation. 

The command can be conveyed in various ways. In one 
embodiment, it can be a trigger bit. in the watermark pay load. 
Normally the bit has a value of u 0." If the bit has a value of" 1 " 
the detector system responds by changing its operation. A 
trigger pattern can also be established, so that detection of a 
certain combination of bits m the watermark pay load serves 
to trigger the change. Reserved states of certain data fields are 
examples of patterns that might be employed. 

Hie command can also be conveyed through another chan- 
nel different than the watermark channel (e.g., an SCA chan- 
nel of an FM broadcast, or the sub-titling data channel of 
video broadcasts, or header data within an MPEG data 
stream, etc., etc.). 

The change can proceed in accordance with a pre-pro- 
grammed rule (e.g., codes progressing successively through a 
numerically or algorithmically- determined progression), or 
the change can proceed in accordance with data specified 
elsewhere in the pay load of the watermark bearing the trigger 
bit (e.g., instead of being interpreted in normal fashion, the 
non-trigger bits of the detected watermark can define a new 
pseudo-random key data. Or the change can proceed in accor- 
dance with data conveyed in successively- presented water- 
mark payloads, as might be done in video encoding where 
each frame of video can convey further watermark informa- 
tion, (This latter arrangement is one offering a high-band- 
width re programming channel through which, e.g., extensive 
firmware instructions might be transferred to the detector to 
replace instructions earlier stored.) 

By such arrangements* greatly increased detector versatil- 
ity and functionality can be achieved. 

Conclusion 

Many diverse embodiments are reviewed above — each 
with a unique set of features. (Still others are disclosed in the 
assignee's patents incorporated by reference.) This specifica- 
tion should be construed as explicitly teaching that features 
illustrated in one such embodiment can generally be used in 
other embodiments as well. Thus, for example, a date field 
was not particularly discussed in connection with payload 
data for video watermarking. Nor were "play once" water- 
marks so-considered. The inclusion of a calibration signal 
with (or as part of) the watermark is shown in embodiments of 
the issued patents, but is not belabored in the above- described 
embodiments. Likewise with "simple universal codes " The 
pre- stored commerce profile described in one of the forego- 
ing embodiments is equally applicable to other embodiments 
as well Likewise, the presentation of advertising was dis- 
cussed in connection with one embodiment but not others, 
although it, too, is generally applicable. All of these concepts 
are familiar at Digimarc find are regarded as generally appli- 
cable throughout the work expressed in Digimarc's patent 


Page 18 of 20 


US 7,562,392 Bl 


33 


34 


disclosures. Practicality prevents an exhaustive recitation of 
each individual permutation and combination. 

Having described and illustrated the principles of our 
invention with reference to illustrative embodiments, it will 
be apparent that the detailed arrangements can be modified in 5 
arrangement and detail without departing from such prin- 
ciples. 

For example, while reference lias been made to various 
uses of wireless, it should be understood that such reference 
does not just cover FM broadcast, and wireless internet net- 10 
working and the like, but also includes other wireless mecha- 
nisms. Examples include cell phones and direct satellite 
broadcast, 

Likewise, while certain embodiments were illustrated with 
a watermark payload of 100+ bits, in other systems much 15 
smaller (or sometimes larger) pay loads are desirable— some- 
times as small as 1-8 bits. 

While the foregoing examples have each been illustrated 
with reference to a particular media type (e.g., video, audio, 
etc), it will be recognized mat the principles of each emhodi- 20 
ment find application with the other media types as well. 

Certain of the appliances contemplated above require user 
interfaces more sophisticated than are presently typical on 
such devices. The simplicity of the underlying audio appli- 
ance can be preserved, in many instances, by using a palmtop 25 
computer — coupled by infrared or otherwise — -as a tempo* 
rary user interface to the appliance. Some of the processing 
capability can likewise be off-loaded to an ancillary palmtop, 
(Palmtop is here meant to refer generally to any pocket-size 
programmable computing device/) *o 

Unless otherwise stated, it should be understood that the 
digital music, video, and imagery contemplated herein is not 
of any particular form or format. Audio, for example, can be 
of various forms, both streaming and non-streaming, and of 
var ious formats (e.g. MP3, MP4, MS Audio, Windows Media 15 
Technologies, RealAudio, *,WAV, MIDI. C sound. Dolby's 
Advanced Audio Codec (AAC), etc. 

To provide a comprehensive disclosure without unduly 
lengthening the present specification applicants incorporate 
by reference the patent publications and applications cited aq 
herein. 

We claim: 

1. A method comprising: 
receiving audio at a device; 

discerning from the audio a plura 1-bit audio ID comprising 45 
a Digital Object Identifier; 

obtaining information from a memory in the device; 

widi reference to at least the information, transmitting at 
least a portion of the audio ID to a location remote from 
said device. 50 

2. A method comprising: 

receiving audio at a device by a microphone, 
discerning from the audio a plural-bit audio ID; 
obtaining a user ID from a memory in the device; S3 
transmitting at least portions of both the audio ID and the 

user ID to a location remote from said device, and 
discerning at least two IDs from the audio, one being said 
audio ID, another being an TD corresponding to an envi- 
ronment in which the device is located, 60 

3. A method comprising: 
receiving audio at a device; 

providing the audio to a processing system; 

receiving from the processing system an audio ID decoded 

from the audio, the audio ID comprises a Digital Object 

Identifier; 

obtaining information from a memory in the device; 


with reference to at least tile information, transmitting at 
least a portion of the audio ID to a location remote from 
said device, 

4. A method comprising: 

receiving audio at a device by a micro phone; 

providing the audio to a processing system; 

receiving from the process ing system an audio ID decoded 
from the audio; 

obtaining a user ID from a memory in the device; 

transmitting at least portions of both the audio H> and the 
user ID to a location remote from said device; 

receiving from the processing system at least two CDs cor- 
responding to the audio, one being said audio ID, 
another being an ID corresponding to an environment in 
which the device is located. 

5. A method comprising: 

receiving electronic signals representing ambient music 

captured with a microphone in a user device; 
transferring the electronic signals representing the ambient 

music to a multi-purpose electronic processor; 
receiving from the multi-purpose electronic processor data 

derived from the electronic signals; 
using said data to obtain information from a database, said 

information relating to the ambient music; 
providing at least textual information to a user about the 

ambient music, said provided textual information being 

based at least in part on information obtained from the 

database, 

providing to the user a po niter to an online site, the online 
she including third-party reviews of content. 

6. The method of claim 5 in which the textual infonnation 
provided to the user specifies the artist and title of the ambient 
music, 

7. The method of claim 5 in which the textual information 
provides the user an opportunity to liave die ambient music, or 
data related thereto, electronically sent to a destination 
device. 

8. The method of claim 7 that further includes the act of 
electronically sending the ambient music* or data related 
thereto, to said destination device. 

9. The method 0 f claim 7 in which the destination device is 
distinct from the user device. 

10. Themetliod 0 f claim 5 in which the textual information 
identifies packaged media on which ihe music is available. 

11. The method of claim 5 in which the user device 
includes a display, and the textual information is presented to 
the user on said display 

12. The method of claim 5 T triggered by a user action 
including pressing a button on the user device, 

13. The method of claim 5 T triggered by a voice command 
of the user, acted upon by a voice recognition feature of the 
user device. 

14. The method of claim 5 in which die device is portable, 
sized to carry in a user's pocket. 

15. The method of claim 5 that includes: 

transmitting data from the user device to a remote com- 
puter, said data including user/device data relating to at 
least one of the following: user name, audio delivery 
infonnation, userage> user gender, model of user device, 
device tJID, or user UID; 

wherein the text presented to the user is dependent, at team 
in part, on said transmitted user/device data. 

16. The method of claim 5 that further includes the act of 
processing the transferred electronic signals to derive the 
data. 

17. The method of claim 16 in which at least some of said 
processing is perfonned hi the user dev ice. 
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18. The method of claim 16 in which the processing com- 
prises decoding a watermark from the transferred electronic 
signals. 

19. The method of claim 5 in which the user device com- 
prises a cell phone. 

20. The method of claim 5 in which the electronic signals 
representing the ambient music are filtered prior to deriving 
the data. 

21 . A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 20. 

22. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose e3ectronic processors to perform the acts of claim 20. 

23. ITie method of claim 5 m which the data comprises an 
identifier 

24. The method of claim 5 in which the pointer comprises 
one or more URL links- 

25 . The method of claim 5 in which the online site provides 
audio associated with the ambient music for a fee. 

26. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to pcrfonn the acts of claim 5. 

27. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 5. 

28. the method of claim 5 in which the multi-purpose 
electronic processor is programmed for deriving data from 
the electronic signals. 

29. A method comprising: 

receiving electronic signals representing ambient music 
captured with a microphone in a user device, the user 
device comprises wireless transmit and receive capabili- 
ties; 

transferring the electronic signals representing the ambient 
music to a multi-purpose electronic processor; 

receiving from the multi-purpose electronic processor data 
derived from the electronic signals; 

using said data to obtain information from a database, said 
information relating to the ambient music; 

providing at least textual information to a user about the 
ambient music, said provided textual information being 
based at least in part on information obtained from the 
database; 

the wireless device has a store- and- forward capability, 
wherein ambient music can be stored and later identified 
if wireless service is not available at the time when the 
ambient music is received by the microphone. 

30. A programmed computing system storing msfructions 
in memory, said instructions- are executable by said pro- 
grammed computing system to perform the acts of claim 29. 

31. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 29. 

32. The method of claim 29 in which the multi-purpose 
electronic processor is programmed for deriving data from 
the electronic signals. 

33 . The method of cla im 29 in which the electronic signals 
representing the ambient music are filtered prior to deriving 
the data. 

34. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perfonu the acts of claim 33. 

35. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 33. 
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36. A method comprising: 

receiving electronic signals representing ambient music 
captured with a microphone hi a user device; 

transferring the electronic signals representing the ambient 
5 music to a multi-purpose electronic processor; 

receiving from the multi-purpose electronic processor data 
derived from the electronic signals; 

using said data to obtain information from a database, said 
information relating to the ambient music; 
io providing at least textual information to a user about the 
ambient music, said provided textual information being 
based at least in part on information obtained from the 
database; and 

providing to the user a pointer to an online site that pro- 
] 5 \ddes streaming media content associated with the ambi- 
ent music. 

37. The method of claim 36 in which the data comprises an 
identifier. 

38. The method of claim 36 in which die pointer comprises 
20 one or more URL links. 

39. The method of claim 36 in wliich the user device 
comprises a cell phone. 

40. The method of claim 36 in which the user device 
includes one or more speakers, and the streaming media con- 

is tent is rendered to a user via at least the one or more speakers. 

41. The method of claim 36 in which the data is derived 
through decoding a watermark from the electronic signals. 

42. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 

10 grammed computing system to perform the acts of claim 36. 

43. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 36. 

44. The method of claim 36 in which the multi-purpose 
,5 electronic processor is programmed for deriving data from 

the electronic si gnals , 

45. The method of claim 36 in which the electronic signals 
representing the ambient music are filtered prior to deriving 
the data. 

40 46. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 45. 

47. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 45. 

48. A method comprising: 

receiving electronic signals representing ambient music 
captured with a microphone in a user device; 
^ transferring the electron ic s i gnai s representing the ambient 
music to a multi-purpose electronic processor; 

receiving from the multi-purpose electronic processor data 
derived from the electronic signals; 

using said data to obtain information from a database, said 
55 information relating to the ambient music; 

providing at least textual information to a user about the 
ambient music, said provided textual information being 
based at least in part on information obtained from the 
database; and 

(50 proinding to the user a pointer to dynamic user metadata 
that is associated with the ambient music. 

49. The method of claim 48 in which the dynamic user 
metadata is updatable over time. 

50. The method of claim 49 in which the dynamic user 
65 metadata comprises artist concert scheduling, 

5T The method of claim 49 in which the dynamic user 
metadata comprises fan-generated metadata. 
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52, The method of claim 48 in which the data comprises an 
identifier. 

53, The method of claim 48 wherein the pointer comprises 
one or URL links. 

54, The method of claim 48 in which the user device 
comprises a cell phone. 

55, The method of claim 48 in which the data is derived 
through decoding a watermark from the electronic signals. 

56, A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 48. 

57, One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 48, 
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58. The method of claim 48 in which the multi-purpose 
electronic processor is programmed for deriving data from 
the elcclronic signals. 

59. The method of claim 48 in which the electronic signals 
5 representing the ambient music are filtered prior to denying 

the data. 

60. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 59. 

ic 61. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 59. 

« n * * * 


